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EXECUTIVE  SUMMARY 


This  report  presents  results  of  the  Phase  II  controlled  site  unexploded  ordnance  (UXO)  advanced  technology 
demonstration  (ATD)  conducted  at  U.S.  Anny  Jefferson  Proving  Ground  (JPG)  in  Madison,  Indiana.  The 
purpose  of  the  UXO  ATD  program  is  to  assess  the  capabilities  of  technologies  useful  for  the  detection, 
i/WifWinn  and  remediation  of  UXO.  Performance  data  gathered  as  a  result  of  the  ATDs  can  be  used  by 
the  Government  to  aid  in  selecting  effective  and  efficient  systems  for  UXO  clearance  in  response  to  the  global 
UXO  problem. 

Problem  Statement  and  Description  of  the  UXO  Clearance  Technology  Program 

In  the  United  States,  millions  of  hectares  of  Government-owned  and  formerly  owned  properties  contain 
buried  UXO.  Bombs,  missiles,  mines,  projectiles,  submunitions,  rockets,  and  other  types  of  ordnance  have 
been  the  result  of  operations  conducted  at  functional  test  ranges,  impact  ranges,  training  areas,  and  disposal 
areas.  Properties  requiring  cleanup  include  installations  identified  for  base  realignment  and  closure  (BRAC), 
formerly  used  defense  sites,  and  active  installations  that  are  considering  alternate  uses  for  properties 
rrtntaining  UXO.  The  Department  of  Defense  (DoD)  has  estimated  that  UXO  cleanup  efforts  in  the  United 
States  alone  may  cost  $28  to  $48  billion. 

In  flHHitinn  more  than  60  countries  report  a  need  to  remediate  a  wide  range  of  UXO,  including  land  mines  and 
ordnance  from  World  Wars  I  and  H.  As  of  December  1994,  the  State  Department  estimated  that  80  to  1 10 
million  land  mines  remain  uncleared,  primarily  in  undeveloped  countries.  This  UXO  results  in  approximately 
10,000  deaths  and  30,000  injuries  each  year.  The  presence  of  UXO  also  denies  countries  the  use  of 
agricultural  land  and  other  resources. 

In  response  to  this  need,  the  U.S.  Army  Environmental  Center  (USAEC)  and  the  Naval  Explosive  Ordnance 
Disposal  Technology  Division  (NAVEODTECHDIV)  established  the  UXO  Clearance  Technology  Program 
to  demonstrate,  evaluate,  and  characterize  advanced  technology  for  UXO  clearance. 
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Summary  of  the  Program  Objective  and  Background 


In  recent  years,  U.S.  Congress  hag  recognized  the  need  for  using  more  effective,  reliable,  safe,  and  economical 
systems  anH  technology  for  UXO  clearance.  In  fiscal  year  1993,  USAEC  efforts  in  demonstrating, 
evaluating,  and  enhancing  UXO  clearance  technology  were  greatly  expanded  to  address  a  congressional 
mandat*  to  demonstrate  and  evaluate  the  performance  of  commercially  available  and  government-enhanced 
systems  designed  for  UXO  clearance. 

In  fiscal  year  1994,  USAEC  created  a  48  hectare  controlled  test  site  containing  inert  ordnance  at  JPG.  The 
test  site  was  comprised  of  a  16-hectare  area  designed  for  ground  demonstrations  and  a  32-hectare  area 
designed  for  airborne  demonstrations.  At  these  two  areas,  inert  ordnance,  nonordnance,  and  debris  were 
buried  at  precise  but  unpublished  locations,  depths,  and  orientations  that  are  representative  of  realistic 
conditions  at  sites  containing  UXO.  The  types  of  ordnance  emplaced  at  the  site  ranged  from  mines  and 
submunitions  less  than  5  inches  in  diameter  to  2,000-pound  bombs.  Ground-based  demonstrators  were 
allotted  40  horns  to  survey  16  hectares,  airborne  demonstrators  were  allotted  40  hours  to  survey  32  hectares, 
anH  remediation  demonstrators  were  allotted  40  hours  to  excavate  as  many  targets  as  possible  during  that 
tims  anH  demonstrate  their  unique  capabilities.  Detection  demonstrator  data  were  then  compared  with 
defined  “target”  locations. 

The  objectives  of  the  1994  Phase  I ATD  project  included  identifying  and  evaluating  commercial,  prototype, 
and  operational  technologies  for  UXO  detection,  identification,  and  remediation;  establishing  a  technology 
performance  baseline;  and  understanding  the  performance  of  current  UXO  clearance  technology.  The 
demonstration  results  also  identified  areas  that  require  further  research  and  development. 

Discussion  of  Phase  II  Objectives,  Process,  and  Demonstrations 

Congress  appropriated  additional  funding  in  fiscal  year  1994  to  continue  the  program  for  a  second  phase.  As 
with  Phase  I,  the  Phase  II  demonstrations  were  conducted  at  JPG.  The  objectives  for  Phase  II  again  included 
evaluating  technologies  effective  for  detecting,  identifying,  and  remediating  UXO,  as  well  as  measuring  these 
results  against  the  Phase  I  baseline.  The  demonstration  areas  were  modified  for  Phase  II,  including  the 
creation  of  a  second  ground  demonstration  area  within  the  air  demonstration  area.  To  ensure  site  integrity  for 
this  phase  of  the  demonstrations,  several  changes  were  made  to  both  areas  as  inert  ordnance  items  were 
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excavated  and  removed  and  new  inert  ordnance  items  were  emplaced.  During  Phase  II,  15  UXO  detection 
systems  sad  two  remediation  systems  were  demonstrated  from  May  through  September  1995,  following 
procedures  similar  to  Phase  I.  Table  ES-1  summarizes  the  detection  technologies  and  transport  systems 
demonstrated  at  the  controlled  site.  Detection  system  technologies  included  four  magnetometer  systems, 
three  electromagnetic  (EM)  induction  systems,  three  ground-penetrating  radar  (GPR)  systems,  and  five  multi¬ 
sensor  systems.  Of  the  detection  systems,  six  were  man-portable,  two  were  vehicle-towed,  four  were 
combined  man-portable  and  vehicle-towed  (multimodal),  and  three  were  airborne.  Two  remediation  systems 
were  demonstrated:  a  remote  backhoe  and  a  soft  trencher.  This  report  summarizes  the  operational 
performance  of  both  detection  and  remediation  systems  based  on  field  oversight  and  their  measured 
performance  based  on  data  analysis. 

Measured  Performance  Results 

After  surveying  their  assigned  area,  each  of  the  demonstrators  was  to  process  their  data  and  report  the 
locations  of  suspected  targets.  For  each  target  location,  the  demonstrators  were  to  supply  estimates  of  the 
depth,  type  (ordnance  or  nonordnance),  size  (small,  medium,  or  large),  and  class  (bomb,  projectile,  mortar,  or 
cluster).  After  receiving  the  demonstrators’  reported  targets,  these  results  were  compared  to  die  baseline  to 
determine  probability  of  detection,  false  alarm  ratio,  mean  radial  and  depth  error,  and  the  fraction  of 
detections  that  were  correctly  identified  or  classified. 

Tables  ES-2  and  ES-3  summarize  demonstrator  performance  with  respect  to  detection,  localization, 

and  classification.  Several  conclusions  can  be  drawn  from  data  analysis  of  the  demonstration 
results.  The  best  performance  in  detecting  ordnance  was  demonstrated  by  Parsons  Engineering  Science,  Inc. 
(Parsons),  using  an  EM  induction  system  and  Geometries,  Inc.  (Geometries),  using  a  magnetometer  system. 
Demonstrators  employing  GPR  exhibited  poor  detection  capabilities.  Systems  employing  a  combination  of 
Tnagnrtrwngtw  and  EM  induction  sensors  had  a  narrow  range  of  high  PD  values  (0.65  -  0.72)  (see 
Table  ES-2).  All  of  the  airborne  demonstrators  showed  poor  detection  capabilities,  regardless  of  sensor  type 
employed. 
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TABLE  ES-1 

DEMONSTRATED  DETECTION  TECHNOLOGIES 


Sensor  System 

GPR 

a 

j 

2 

tu 

i 
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Magnet¬ 

ometer 

a 

. 

i 

Transport 

Aerial 

Fixed- 
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Rotary- 

Wing 

* 

Ground 

Man- 

Portable 

B 

i 

i 

Vehicle- 

Towed 

B 

■ 

Demonstrator 

Aerodat  Inc. 

Airborne  Environmental  Surveys,  Inc. 

Australian  Defence  Industries,  Pty.  Ltd. 

Bristol  Aerospace  Ltd. 

Coleman  Research  Corporation 

Geo-Centers,  Inc. 

Geometries,  Inc. 1 

Geophex  Ltd. 

C 

I 

8 

J2 

jlCaman  Sciences  Corporation 

Parsons  Engineering  Science,  Inc. 

Polestar  Technologies,  Inc. 

Scintrex,  Inc. 

jjsRI  International 

X 

i 

i 
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Note:  1  Geometrics’s  GPR  system  was  demonstrated  in  the  field,  but  results  were  not  included  for  analysis. 
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NA  Not  applicable  because  no  ordnance  items  were  detected 
1  False  alarms  per  UXO  items  detected 


TABLE  ES-3 

FRACTION  OF  DEMONSTRATOR  TARGET  REPORTS  IDENTIFIED  AND  CLASSIFIED 
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NA  Not  applicable 

1  Demonstrator  reported  all  target  types  as  “ordnance.’ 


The  false  alarm  ratio,  defined  as  the  number  of  false  alarms  divided  by  the  number  of  ordnance  items 
(tetpp-ted,  assesses  the  impact  of  a  demonstrator’s  false  alarms  on  a  remediation  effort.  The  ratio  defines  the 
number  of  nonordnance  items  that  would  be  excavated  for  every  ordnance  item.  False  alarm  ratios  few*  the 
Phase  II  demonstrators  ranged  from  approximately  3  to  70,  differing  greatly  between  the  various  systems. 
Although  the  GPR  systems  had  the  lowest  false  alarm  ratios,  this  was  outweighed  by  their  low  detection 
capabilities  (PD  values).  The  highest  false  alarm  ratio  was  produced  by  a  ground-portable  magnetometer 
system. 


Figure  ES-1  presents  a  plot  of  PD  versus  false  alarm  ratio  for  the  demonstrators.  The  plot  shows 
demonstrators  with  better  detection  performance  in  the  upper  left  comer,  and  those  with  poorer  performance 
in  the  lower  right  comer.  Both  of  the  best  performers  (Parsons  and  Geometries)  employed  advanced  data 
processing,  which  may  have  contributed  to  their  good  performance. 


Figure  ES-2  illustrates  the  localization  capability  of  the  various  sensor  types.  The  figure  depicts  the  mean 
horizontal  positions  and  mean  vertical  depth  errors  in  meters.  With  the  exception  of  airborne  systems, 
magnetometer  sensors  and  a  combination  of  magnetometer  and  EM  induction  sensors  proved  to  have  the  best 
localization  capability. 


With  respect  to  identification  (classification  by  type),  few  demonstrators  showed  any  capability  to 
discriminate  between  ordnance  and  nonordnance  (see  Table  ES-3).  Geophex  Inc.,  had  the  best  identification 
performance,  with  a  0.23  probability  of  correctly  classifying  nonordnance  (Figure  ES-3).  Most 
demonstrators  providing  class  information  were  more  successful  at  correctly  identifying  bombs  and 
projectiles  than  other  ordnance  classes.  Although  many  of  the  demonstrators  detected  clusters,  none 
exhibited  the  ability  to  correctly  classify  them.  It  is  possible  that  the  resolution  of  their  systems  or  processing 
is  not  sufficient  to  resolve  closely  spaced  clusters  of  ordnance  items. 


The  two  remediation  demonstrators  were  successful  at  excavating  targets  within  the  given  24  hours.  The 
remote  excavation  vehicle  system  demonstrated  an  average  rate  of  0.57  hour  per  hole  including  ordnance 
removal.  The  soft  trencher  demonstrated  an  average  rate  of  0.75  hour  per  hole  without  ordnance  removal,  at 
depths  ranging  from  0.03  to  3.8 1  meters. 
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Field  Operations 


The  capabilities  and  limitations  of  the  various  detection  systems  were  influenced  by  the  transportation  mode 
utilised  the  terrain  at  the  demonstration  areas,  and  the  weather  conditions.  Man-portable  systems  were  more 
durable  and  were  able  to  access  the  entire  site  successfully;  however  these  systems  were  limited  by  the  speed 
and  gtamina  of  the  field  equipment  operator.  Vehicle-towed  systems  covered  the  site  quickly  but  were  often 
subject  to  breakdowns  that  caused  time-consuming  delays.  Multimodal  systems  were  able  to  easily  traverse 
open  areas  with  vehicle  systems  and  use  man-portable  platforms  in  heavy  vegetation  areas.  Airborne 
systems,  while  yielding  the  best  coverage  rates,  proved  to  have  veiy  low  detection  capabilities.  Both 
remediation  systems  were  prone  to  frequent  breakdowns. 

Summary 

Currently  available  site  characterization  and  remediation  tools  are  not  adequate  to  effectively  and 
economically  respond  to  the  UXO  problem.  First,  although  survey  rates  for  the  more  effective  systems  are  on 
the  order  of  0.40  hectares  per  hour,  a  significant  amount  of  ordnance  will  remain  undetected.  Even  the  best 
system  during  Phase  H  faded  to  detect  15  percent  of  the  ordnance  items.  Second,  the  inability 

to  distinguish  nonordnance  from  ordnance  results  in  high  false  alarm  ratios;  this  means  that  most  of  the  effort 
to  excavate  buried  UXO  will  be  nonproductive.  For  example,  some  of  the  better  performers  had  false  alarm 
ratios  ranging  from  3.4 1  to  4.68.  Even  with  these  results,  for  every  excavation  producing  a  UXO  item, 
approximately  three  to  five  holes  would  be  excavated  that  would  not  contain  UXO.  Finally,  the  excavation 
process  itself  is  a  limiting  factor  to  remediation  as  detection  systems  can  locate  suspected  targets  up  to  20 
time*  faster  than  they  can  be  excavated. 

As  it  becomes  necessary  to  address  the  growing  number  of  UXO  sites  worldwide,  more  capable  technologies 
will  be  needed  to  assist  in  site  restoration  and  risk  reduction  processes.  The  UXO  ATD  project  constitutes  an 
important  first  step  in  a  series  of  testing  and  evaluation  projects  to  help  meet  this  need.  The  two  phases  of 
this  project  have  identified  key  technology  issues  and  lessons  learned  that  need  to  be  addressed  to  meet  DoD 
needs: 


PD  must  be  improved  while  a  reduction  in  false  alarms  is  realized. 
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•  False  alarms  must  be  reduced  through  more  sophisticated  processing  and  better 
discrimination  techniques. 

•  Classification  capabilities  must  be  improved  to  make  remediation  more  cost-effective. 

•  Advanced  data  processing  and  data  fusion  from  multi-sensor  platforms  must  be  investigated. 

•  Site  survey  speed  must  be  increased  without  compromising  detection  capability. 

•  To  be  useful,  large  area  survey  technologies  such  as  airborne  systems  must  be  unproved. 

•  Remediation  rates  must  be  improved  in  order  to  make  facility  cleanup  possible. 

•  New  approaches  for  remediating  surface  and  subsurface  UXO  are  needed. 

•  Systems  need  to  be  evaluated  under  varying  environmental  and  topographic  conditions. 

•  Systems  to  be  evaluated  according  to  specific  performance  capabilities  and  strengths. 


The  Phase  I  results  showed  that  there  was  very  limited  detection  capability,  excessively  high  false  alarm  rates, 
and  minimal  Hicrrimination  and  classification  ability.  Of  the  25  systems  tested  during  Phase  I  of  the  UXO 
ATDs,  the  best  technology  could  only  detect  approximately  65  percent  of  the  inert  ordnance  emplaced  at  the 
controlled  test  site.  The  achieve  this  detection  rate,  however,  the  system  reported  nearly  60  false  alarms  per 
UXO  item  detected,  and  demonstrated  no  ability  to  discriminate  between  UXO  and  clutter  and  debris. 


Phase  II  results  showed  some  improvement  as  the  best  systems  detected  approximately  85  percent  of  the 
emplaced  ordnance  and  reported  about  4  false  alarms  per  UXO  item  detected.  This  significant  increase  in 
detection  and  corresponding  reduction  in  false  alarms  from  Phase  I  to  Phase  II  was  principally  attributed  to 
the  proper  selection  and  application  of  technology  for  the  JPG  environment  However,  as  in  Phase  I,  none  of 
the  systems  demonstrated  any  significant  ability  to  classify  UXO  or  discriminate  between  UXO  and  clutter 
and  debris. 
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Figure  ES-1 

Plot  of  Demonstrator  Probability  of  Detection  (Ordnance)  versus  False  Alarm  Ratio 
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False  Alarm  Ratio  (False  Alarms/Ordnance  Items  Detected) 


Figure  ES-2 

Individual  Demonstrator  Localization  Performance 
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□  Horizontal  Position  Error  (Radial)  ■  Absolute  Depth  Error 


Figure  ES-3 

Individual  Demonstrator  Probability  of  Correct  Identification  (Gassification  by  Type) 
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Notes:  1  Demonstrator  reported  all  target  declarations  as  “ordnance” 

2  Demonstrator  did  not  provide  type  information  for  declarations  or  listed  as  “unknown’ 


1.0  INTRODUCTION 


Many  hectares  of  previously  owned  or  currently  owned  U.S.  Department  of  Defense  (DoD)  sites  contain 
unexploded  ordnance  (UXO)  as  a  result  of  military  training  and  testing  activities  conducted  to  maintain 
mission  readiness.  The  types  of  ordnance  at  these  sites  range  from  centuries-old  cannonballs  to  currently 
used  rockets,  projectiles,  bombs,  mortars,  submunitions,  and  mines. 


The  U.S.  Army  Environmental  Center  (USAEC)  has  established  and  currently  manages  the  UXO  Clearance 
Technology  Program  to  address  UXO  issues.  The  goal  of  this  program  is  to  enhance,  demonstrate,  and 
evaluate  UXO  identification  and  remediation  technologies  to  provide  the  Government  with  more  reliable, 
accurate,  safe,  and  cost-effective  methods  for  UXO  characterization  and  clearance.  USAEC  has  designated 
the  Naval  Explosive  Ordnance  Technology  Division  (NAVEODTECHDIV)  as  the  technical  lead  for  this 
program. 


1.1  PROGRAM  BACKGROUND 

In  response  to  the  congressional  mandate  in  HR  5504,  USAEC  and  NAVEODTECHDIV  established  the 
Advanced  Technology  Demonstration  (ATD)  program  to  identify  and  evaluate  technologies  for  UXO 
detection,  identification  and  remediation  (U.S.  House  of  Representatives  [USHR]  1992a,  1992b,  1992c).  In 
fiscal  year  1994,  USAEC  and  NAVEODTECHDIV  created  two  controlled  demonstration  sites  at  the 
Jefferson  Proving  Ground  (JPG)  in  Madison,  Indiana.  Phase  I  of  the  UXO  ATD  program  was  conducted 
from  April  through  October  1994;  Phase  ff  was  conducted  from  May  through  September  1995. 

In  preparation  for  the  Phase  I  demonstrations,  two  controlled  test  site  areas  were  prepared:  a  16-hectare 
(40-acre)  area  for  ground  system  demonstrations  and  a  32-hectare  (80-acre)  area  for  airborne  system 
demonstrations.  These  areas  are  referred  to  as  “controlled”  because  the  emplaced  items  provide  a  known 
baseline  for  the  performance  measurement  of  clearance  technologies.  The  areas  were  surveyed,  and  a 
30.5-  by  30.5-meter  (100-  by  100-foot)  grid  system  was  established  in  each  area.  The  areas  were  then 
prepared  by  emplacing  inert  ordnance  and  nonordnance  items  at  depths  and  orientations  typical  ofUXO- 
contaminaied  areas.  The  position  of  each  emplaced  item  was  measured  by  a  licensed  surveyor  and  recorded 
in  a  target  database  to  provide  a  baseline  against  which  demonstrator  performance  could  be  measured. 
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For  Phase  1, 29  systems  were  demonstrated  from  April  through  October  1994  (USAEC  1995).  The  systems 
included  man-portable,  vehicle,  combination  man-portable  and  vehicle  (multi-modal),  airborne, 
and  remediation  systems.  In  addition,  a  variety  of  sensor  technologies  were  demonstrated,  including 
mzgfjetomcter  systems,  electromagnetic  induction  systems,  ground  penetrating  radar  systems,  infrared 
systems,  and  combinations  of  the  above. 

Phase  II  was  open  to  all  technology  developers.  Demonstrators  who  participated  in  Phase  I  were  eligible  for 
Phase  H  only  if  significant  improvements  or  changes  had  been  made  to  their  system.  Proposals  were 
evaluated  in  a  manner  similar  to  Phase  I;  selection  criteria  included  technical  approach,  experience,  and  best 
value  to  the  Government  A  total  of  17  systems  were  accepted  by  the  government  panel  for  Phase  II. 

Phase  II  was  conducted  from  May  through  September  1995.  The  17  systems  demonstrated  in  Phase  II 
included  three  airborne  systems,  six  man-portable  systems,  two  vehicle-towed  systems,  four  combined  man- 
portable  and  vehicle-towed  (multimodal)  systems,  and  two  remediation  systems.  The  15  detection 
technologies  used  magnetometer  systems,  electromagnetic  induction,  and  ground  penetrating  radar  systems, 
as  shown  in  Table  1-1.  This  report  discusses  Phase  II  of  the  UXO  detection,  identification,  and  remediation 
ATDs. 

To  ensure  data  uniformity  each  demonstrator  was  requested  to  submit  data  in  a  standardized  data  entry 
software  program  Measures  of  effectiveness  were  developed  to  provide  a  technically  meaningful  framework 
for  assessing  demonstrator  performance.  The  measures  were  based  on  a  target-matching  algorithm  (TMA) 
developed  for  this  project  and  were  expressed  as  target  detection  ratios  (percentages  of  emplaced  targets 
located  by  each  demonstrator);  classification  ratios  (percentages  of  emplaced  targets  correctly  identified  by 
each  demonstrator);  error  ratios  (percentages  of  each  demonstrator’s  reported  targets  declared  to  be 
incorrectly  identified  ordnance);  and  additional  data,  such  as  the  size,  class,  depth,  and  relative  orientation  of 
each  target.  Demonstrator  Hata  were  collected,  entered  into  the  target  database,  and  analyzed  using  the  TMA. 
Phase  I  results  were  presented  in  Report  No.  SFTM-AEC-ET-  CR-94120  dated  December  1994.  Ground 
based  system  overall  detection  ratios  ranged  from  1  to  65  percent  Airborne  systems  received  overall 
detection  ratios  between  0  and  8  percent.  The  remediation  systems  demonstrated  during  Phase  I  operated  at  a 
slow  rate  (between  4  and  1 1  targets  in  40  hours),  but  proved  successful  in  excavating  targets. 
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TABLE  1-1 

DEMONSTRATED  TECHNOLOGIES 


Demonstrator 

Magnetometer 

Electromagnetic 

Induction 

Ground  Penetrating 
Radar 

Airborne  Systems 

Aerodat  Inc. 

X 

Airborne  Environmental 

Surveys,  Inc. 

X 

SRI  International 

X 

Man-Portable  Systems 

Australian  Defence 

Industries,  Pty.  Inc. 

X 

X 

Geophex  Ltd 

X 

X 

GeoPotential 

X 

Parsons  Engineering 

Science,  Inc. 

X 

Polestar  Technologies,  Inc. 

X 

Scintrex,  Inc. 

X 

Vehicle  Systems 

Bristol  Aerospace  Ltd. 

X 

Kaman  Sciences  Corp. 

X 

Multimodal  Systems 

Coleman  Research  Corp. 

X 

X 

Geo-Centers,  Inc. 

X 

X 

Geometries,  Inc. 

X 

X 

Vallon  GmbH  (in  cooperation 
with  Security  Search  Products) 

X 
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DEMONSTRATION  OBJECTIVES 


12 

The  objectives  of  the  Phase  II UXO-ATD  program  were  to  obtain  system  capability  and  performance  data  on 
UXO  detection  anH  remediation  technologies.  The  data  objectives  for  demonstrators  were. 


•  to  report  the  location  of  targets  detected  in  the  test  area 

•  to  localize  targets  detected  within  the  test  area 

•  to  discriminate  ordnance  from  nonordnance  targets 

•  to  classify  ordnance  targets  detected  in  the  test  area 

•  and  in  the  case  of  remediation  demonstrators,  excavate  or  remove  ordnance  items. 


The  objectives  of  the  evaluators  of  the  Phase  II  data  were: 


•  to  evaluate  overall  system  and  technology  performance  and  provide  system  capability  and 
limitation  information  to  the  user  community 

•  to  evaluate  individual  sensor  performance,  providing  useful  information  for  demonstration 
coni  actors  and  end  users  of  the  technology,  as  well  as,  potentially  indicate  directions  for 
future  sensor  technology  improvement 

•  to  compare  system  performance  against  the  Phase  I  baseline 

•  to  identify  promising  technologies  that  can  be  used  to  provide  more  economical,  safe,  and 
effective  UXO  clearance 

•  to  identify  research  and  development  efforts  that  would  provide  the  Government  with  the 
greatest  return  on  investment 


1-4 


2.0  SITE  BACKGROUND 


2.1  SITE  DESCRIPTION 

JPG  is  located  about  5  miles  north  of  Madison,  Indiana,  in  Jefferson,  Ripley,  and  Joinings  Counties 
(See  Figure  2-1).  The  facility  covers  about  22,365  hectares  (55,265  acres)  and  includes  firing  lines,  impact 
areas,  buildings,  and  roadways. 

As  in  Section  1.1,  two  areas  within  JPG  woe  selected  for  demonstrations  during  Phase  I  of  the 

UXO-ATD  program.  A  16-hectare  area  in  the  northwest  quarter  of  Section  36,  Township  6  North,  Range 
10  East  was  designed  for  demonstrations  of  ground  systems.  A  32-hectare  area  at  the  center  of  Section  14, 
Township  5  North,  Range  10  East  was  designed  for  demonstrations  of  airborne  systems.  The  areas  are 
located  adjacent  to  access  roads  on  the  east  side  of  the  facility  (see  Figure  2-2). 

2.1.1  Topographic,  Physiographic,  and  Geologic  Properties 

Topographic  relief  in  Jefferson  County  is  influenced  by  the  Ohio  and  Muscaiatuck  River  watersheds.  The 
Ohio  River  watershed,  located  in  the  eastern  third  of  Jefferson  County,  is  very  dissected  and  is  characterized 
by'  narrow,  sloping  ridges  and  steep  hillsides  with  terraces.  The  Muscatatuck  River  watershed,  located  in  the 
western  two-thirds  of  Jefferson  County,  in  characterized  by  broad,  nearly  level  ridges  and  moderately  sloping 
hillsides.  The  major  tributary  of  the  Ohio  River  in  Jefferson  County  is  Indian-Kentuck  Creek,  which  drains 
the  eastern  third  of  the  county  (USDA  1985b). 

Physiographically,  the  demonstration  areas  are  nearly  level  with  a  slightly  undulating  surface,  marked  by 
minor  erosional  features  from  surface  water  runoff.  Both  areas  are  well-vegetated  with  grasses,  shrubs,  and 
trees.  No  tributaries  to  the  Ohio  or  Muscatatuck  Rivers  dissect  the  demonstration  areas. 

The  demonstration  areas  are  located  on  the  uplands,  in  areas  of  sparse  forestation.  Both  areas  are  located 
adjacent  to  access  roads  along  the  east  side  of  the  facility.  Drainage  at  the  16A-hectare  area  is  to  the  west 
into  Big  Creek.  Drainage  at  the  32-hectare  area  is  to  the  east  into  West  Fork  Creek. 
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Surficial  soils  are  situated  on  a  flat  plain  known  as  the  Illinoisan  till  plain  (Indiana  Department  of  Natural 
Resources  [IDNR]  no  date).  The  plain  consists  of  glacial  till  deposited  during  Illinoisan  glaciation.  The 
glacial  deposits  are  underlain  by  Silurian-aged  Laurel  Dolomite  bedrock.  The  Laurel  Dolomite  is  about 
14  meters  (45  feet)  thick,  gray,  and  cherty.  Below  the  bedrock,  Silurian-  and  Ordovician-aged  interbedded 
limestone  and  shale  extends  from  91  to  121  meters  (300  to  400  feet).  Depth  to  bedrock  at  the  demonstration 
areas  ranges  from  1.5  to  9  meters  (5  to  30  feet)  below  ground  surface  (PRC  1994). 

Native  soils  at  the  16-hectare  and  the  32-hectare  areas  consist  mainly  of  Avonburg  and  Cobbsfork  silt  loams. 
Avonburg  soils  are  nearly  level,  deep,  and  somewhat  poorly  drained  soils  situated  on  smooth  uplands.  Areas 
of  this  soil  type  are  broad  and  irregular  in  shape  and  cover  8  to  80  hectares  (20  to  200  acres)  (USDA  1985b). 
Cobbsfork  soils  are  nearly  level,  deep,  and  poorly-drained  soils  situated  on  tabular  divides  in  uplands; 
Cobbsfork  soils  are  prone  to  ponding.  Areas  of  this  soil  type  are  broad  and  irregularly  shaped,  ranging  from 
16  to  8 10  hectares  (40  to  2,000  acres)  in  size  (USDA  1985a). 

Cobbsfork  soils  have  a  very  high  available  water  capacity  and  very  slow  permeability.  Avonburg  soils  have  a 
ni<vfr»rat*  available  water  capacity  and  very  slow  permeability.  In  both  soil  types,  the  water  table  is  typically 
perched  at  or  near  the  surface  during  most  of  the  year.  Both  the  Avonburg  and  Cobbsfork  soils  are  low  in 
organic  matter,  and  they  are  acidic,  friable,  and  best  suited  for  grass  and  tree  development  (USDA  1985a  and 

1985b). 

2.1.2  Ecologic  Characteristics 

JPG  consists  primarily  of  poorly  drained  flats  in  various  stages  of  succession  from  open  fields  to  regrowth 
forested  flatwoods.  Flatwoods  are  forested  areas  that  occur  on  level  or  nearly  level  soils  that  are  poorly 

drained  and  have  a  shallow  perched  water  table.  Some  wooded  stream  valleys  with  better  drainage  are  also 

present  at  JPG.  Vegetative  community  types  that  have  been  inventoried  by  the  IDNR  Division  of  Nature 
Preserves  include  bottomland  forests,  upland  forests,  and  cliffs  along  these  major  drainages  (IDNR  no  date). 

JPG  lies  within  the  Bluegrass  Natural  Region,  as  identified  by  IDNR  This  natural  region  is  identified  and 
nanvH  for  its  similarities  in  physiography  and  natural  communities  to  the  Bluegrass  Region  of  Kentucky. 
Most  of  the  natural  region  was  originally  forested,  although  a  few  glade,  cliff,  and  barren  remnants  are 
known,  as  well  as  nonforested  aquatic  communities.  The  areas  used  for  the  UXO  demonstrations  can  be 
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classified  as  Bluegrass  Till  Plain  Flatwoods.  These  natural  communities  are  forested  areas  on  level  or  nearly 
level  soils  that  are  poorly  drained  and  acidic,  with  a  shallow  perched  water  table  (IDNR  no  date). 

.  2.13  Compliance  with  Environmental  Regulations 

On  February  25, 1994,  NAVEODTECHDIV  and  JPG  signed  a  Record  of  Environmental  Consideration 
(REC)  in  accordance  with  the  environmental  regulations  of  Army  Regulation  200-2  and  the  National 
Environmental  Policy  Act  The  REC  provided  a  categorical  exclusion  for  the  creation  of  the  controlled  site  at 
JPG  and  for  its  use  in  the  demonstrations.  Wetlands,  endangered  species,  and  archaeological  investigations 
were  conducted  as  part  of  the  REC.  The  REC  states  that  no  effect  on  these  resources  is  expected  as  a  result 
of  the  demonstrations;  therefore,  no  further  environmental  documentation  was  required.  Because  the  Phase  II 
activities  were  similar  in  nature  and  planned  for  the  same  locations  as  those  for  Phase  I,  the  REC  was 
determined  to  apply  to  Phase  II  as  well. 

2.1.4  Climatic  Patterns 

Climate  in  Jefferson,  Ripley,  and  Jennings  Counties  is  cold  in  winter  and  hot  in  summer.  Winter  precipitation 
consists  mainly  of  snow,  which  aids  in  soil  moisture  accumulation  and  minimizes  drought  conditions  in 
summer  months.  In  winter,  the  average  temperature  is  about  2°C  (35  °F);  the  average  daily  minimum 
temperature  is  about  -4 °C  (25  °F).  In  summer,  the  average  temperature  is  about  24°C  (75  °F);  the  average 
daily  maximum  temperature  is  about  30  °C  (85  °F). 

The  total  annual  precipitation  is  about  107  centimeters  (42  inches),  with  about  55  centimeters  (22  inches) 
falling  from  April  through  September.  Thunderstorms  occur  about  50  days  per  year;  tornados  and  severe 
weather  also  occur  occasionally.  These  types  of  storms  are  usually  local  and  short  in  duration  and  can  cause 
severe  damage  locally  (USDA  1985a,  1985b). 

The  average  seasonal  snowfall  is  about  33  centimeters  (13  inches).  The  average  relative  humidity  in 
midafteraoon  is  about  60  percent.  Humidity  is  higher  at  night,  and  the  average  at  dawn  is  about  80  percent. 
The  sun  shines  70  percent  of  the  time  in  summer  and  40  percent  in  winter.  Prevailing  winds  are  from  the 
south.  Average  wind  speed  is  highest  in  spring  at  16  kilometers  (10  miles)  per  hour  (USDA  1985a,  1985b). 
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HISTORIC  SITE  USE 


An  extensive  survey  of  historical  data  related  to  the  site  indicates  that  fanning  was  the  predominant  land  use. 
The  land  was  typified  by  relatively  small,  dispersed  farmsteads  and  communities.  Both  woodland  and 
agricultural  tracts  occurred  in  the  two  controlled  site  areas.  In  1940,  the  federal  government  acquired  the 
land;  the  first  round  of  ammunition  was  tested  at  JPG  on  May  10, 1941  (USAEC  1995). 

As  part  of  the  background  investigation  for  the  Phase  I UXO  ATD  program,  an  archaeological  investigation 
was  performed  in  November  1993.  This  study  revealed  that  both  areas  were  used  for  agricultural  purposes 
before  the  federal  government  acquired  the  land.  One  site  identified  at  the  32-hectare  area  was  believed  to  be 
a  historic  farmstead  that  was  abandoned  in  194 1 .  The  farmhouse  was  moved  from  the  JPG  property  to  the 
east,  along  Highway  42 1 .  Two  other  sites  identified  in  the  study  were  of  indeterminate  historic  affiliation. 
None  of  the  sites  identified  were  eligible  for  listing  on  the  National  Register  of  Historic  Places  (Ans linger 
1993). 

Geophysical  and  geotechnical  surveys  were  conducted  in  1994  to  establish  area  conditions  and  identify 
hazardous  conditions.  These  surveys  were  conducted  as  part  of  the  preparation  for  Phase  I  of  the  UXO  ATD 
program.  The  survey  results  identified  no  hazardous  conditions  to  preclude  the  use  of  these  areas  as 
controlled  demonstration  areas.  However,  given  the  nature  and  mission  of  JPG,  a  considerable  amount  of  the 
total  base  area  has  undoubtably  been  affected  by  munition  testing  and  related  activities. 

Until  September  1995,  JPG  served  as  a  munitions  testing  facility  of  the  Test  and  Evaluation  Command,  U.S. 
Army  Material  Development  and  Readiness  Command.  During  the  period  of  operation,  JPG  s  mission  was 
to  check,  investigate,  and  evaluate  various  test  items  to  determine  whether  they  conformed  to  specifications 
(JPG  1980).  Between  1942  and  1995,  JPG  conducted  a  variety  of  munitions  tests  throughout  the  base. 
Although  neither  of  the  areas  used  for  the  controlled  site  are  specifically  located  where  these  tests  took  place, 
they  may  be  within  the  “fan”  area  of  several  of  the  impact  fields.  The  controlled  site  areas  are  believed  to 
have  been  only  minimally  affected  by  historical  activities  conducted  at  JPG. 
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SCALE:  1”  =  10,000' 

SOURCE:  MODIFIED  FROM  USGS.  30  X  60  MINUTE 
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3.0  DEMONSTRATION  TECHNICAL  APPROACH 
3.1  SITE  PREPARATION 

The  sites  were  prepared  for  Phase  I  of  the  controlled  site  UXO  ATD  by  mowing  the  tall  grass,  herbs,  and 
woody  vegetation  that  covered  both  demonstration  areas.  Some  small  sapling  trees  and  brush  thickets 
remained.  A  JPG  survey  crew  then  surveyed  both  areas  using  a  total  station  survey  instrument  A  grid 
system  was  established  at  both  areas  cm  a  30.5-  by  30.5-meter  (100-  by  100-foot)  pattern.  A  stake  was 
driven  at  each  grid  node  to  mark  the  comer  locations.  Each  stake  was  marked  with  die  grid  node's 
alphanumeric  code.  A  piece  of  5-centimeter  (2-inch)-diameter  polyvinyl  chloride  (PVC)  pipe  about 
30  centimeters  (12  inches)  long  was  driven  into  the  ground  at  each  grid  node  until  it  was  flush  with  the 
ground  surface.  The  PVC  pipes  were  used  to  hold  the  stakes  in  place.  Ground  surface  elevation  was 
measured  to  the  closest  3  centimeters  (about  0. 1  foot)  at  each  grid  node  and  recorded. 

Permanent  benchmarks  (or  first  order  survey  monuments)  were  established  at  each  area  to  provide  precise 
positioning  aids  for  target  emplacement  by  the  government  and  for  target  location  by  die  demonstrator. 

Three  geodetic  benchmarks  were  established  within  the  16-hectare  area,  and  four  were  established  at  the 
32-hectare  area.  The  node  stakes  were  also  used  as  benchmarks  to  measure  elevations  and  locations  of  inert 
ordnance  and  debris  items  during  emplacement. 

After  the  areas  were  mowed  and  the  monuments  were  established,  an  aerial  survey  was  performed  to  collect 
topographic  measurements  and  identify  locations  of  vegetation  and  terrain  features.  Elevation  data  points 
were  established  throughout  and  around  the  edges  of  each  grid  so  that  a  detailed  contour  model  of  each 
demonstration  area  could  be  constructed.  Topographic  maps  of  both  areas  were  produced  for  use  by  the 
demonstrators. 

Inert  ordnance  and  nonordnance  items  comprising  the  baseline  target  set  were  then  emplaced  at  both 
demonstration  areas.  The  areas  were  designed  to  simulate  three  different  UXO  scenarios:  a  militaiy  training 
area,  an  ordnance  disposal  site,  and  a  formerly  used  defense  site.  A  small  sample  of  mines  were  also 
emplaced  in  both  areas  to  assess  the  plastic  mine  detection  capabilities  of  the  various  systems.  The  table 
below  lists  the  types  of  inert  ordnance  emplaced  at  the  two  areas. 
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Ordnance 

Classification  Size 

Range  of  Emplacement 
depths  (meters) 

Bombs 

2,000-pound  (lb),  1,000-lb,  750-lb,  500-lb,  and  250-lb 

0.15-5.37 

Projectiles 

8-inch,  175-millimeter  (mm),  155-mm,  152-mm,  106- 
mm  105 -mm,  90-mm  armor-piercing  and  high  explosive 
anti-tank  (HEAT),  and  76-mm  HEAT 

0.22-4.17 

Rocket  warheads 

5 -inch  and  2.75-inch 

0.15-0.47 

Mortars 

4.2-inch,  8 1-mm,  and  60-mm 

0.01  - 1.43 

Submunitions 

M-42  annex’  defeating  bomblets 

0  -  0.87 

Landmines 

TS-50  and  VS-50  antipersonnel  mines 

0  -  0.04 

Aircraft  cannon 

30-mm  and  20-mm  rounds 

0.05  - 1.83 

Before  inert  ordnance  were  emplaced,  research  was  conducted  to  determine  realistic  depths  and  orientations 
for  the  ordnance  items.  The  following  guidelines  were  used  to  emplace  ordnance: 

•  General  purpose  bombs  (250  to  2,000  lb)  have  been  found  at  depths  exceeding  6  meters  (m) 
(20  ft),  at  no  predicable  orientation  to  the  surface  plane. 

•  Projectiles  (76  mm  to  8  inch)  are  typically  found  at  depths  from  0.3  to  3.7  meters  (m)  (1  to 
12  ft);  orientation  is  horizontal  to,  or  at  a  slight  angle  from,  die  surface  plane. 

•  Air-launched  rockets  (2.75  and  5  inch)  are  generally  found  at  depths  of  1  to  2.4  m  (3  to  8  ft); 
orientation  is  typically  between  45  and  90  degrees  from  the  plane  of  the  surface  due  to  the 
angle  of  trajectory. 

•  Mortar  rounds  are  generally  found  within  120  cm  (48  in)  of  the  surface;  orientation  is 
typically  between  45  and  90  degrees  from  the  plane  of  the  surface  due  to  the  high  angle  of 
trajectoiy. 

•  Submunitions  are  generally  small,  with  no  standard  size  or  shape,  and  are  dispensed  from 
cluster  bombs  or  artillery  rounds;  they  are  generally  found  on  die  surface,  although  they  may 
be  buried  by  secondary  explosions. 

•  20  mm  and  30  mm  aircraft-  and  ground-delivered  flat  trajectory  gunfire  typically  results  in 
the  projectile  penetrating  the  ground  no  more  than  30.5  centimeters  (cm)  (12  in),  resting 
horizontal  to  the  plane  of  the  ground  surface. 
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Sources  of  false  alarms  (e.g.,  fragments,  building  materials,  and  scrap  metal)  were  also  emplaced  at  the 
controlled  site.  In  addition  to  the  inert  nonordnance  items,  other  sources  of  false  alarms  were  also  introduced 
For  example,  many  active  sensors  can  detect  disturbed  soil,  therefore,  some  holes  were  dug  and  the  soil 
replaced  without  emplacing  a  target  All  of  the  locations  were  surveyed  and  recorded.  After  all  targets  were 
emplaced  and  surveyed,  both  demonstration  areas  were  tilled  and  reseeded  to  present  a  uniform  appearance 
and  prevent  visual  detection  by  the  demonstrators. 

A  demonstrator  reference  area  (DRA)  was  also  established  near  the  16-hectare  area.  Four  inert  ordnance 
items  were  emplaced  in  the  comers  of  the  DRA.  The  ordnance  types,  locations,  and  depths  were  given  to  the 
demonstrators  to  enable  them  to  calibrate  their  systems. 

Several  changes  were  made  in  the  UXO  ATD  program  after  the  completion  of  Phase  I.  To  ensure  site 
integrity  for  Phase  II,  the  demonstration  areas  were  modified,  including  the  creation  of  a  second  16-hectare 
ground  demonstration  area  within  the  boundaries  of  the  32-hectare  area.  The  original  16-hectare  area  is 
referred  to  as  16A-hectare  area  to  differentiate  it  from  the  second  16B-hectare  area  created  for  Phase  II 
ground  demonstrations. 

In  spring  1995,  several  changes  were  made  to  the  16A-hectare  area.  Numerous  inert  ordnance  items 
originally  emplaced  for  Phase  I  of  the  UXO  ATD  program  were  excavated  and  moved  to  new  locations.  As 
ordnance  items  were  excavated  and  removed,  the  locations  were  noted.  In  addition,  a  number  of  new  inert 
ordnance  items  were  emplaced  at  new  locations.  Once  these  changes  were  made  at  the  1 6  A-hectare  area,  a 
new  target  map  was  produced 

The  16B-hectare  area  was  established  for  returning  ground-based  demonstrators  by  using  part  of  the  north 
end  (7.2  hectares)  and  part  of  the  south  end  (8.8  hectares)  of  the  32-hectare  area,  for  a  total  area  of  about 
16  hectares  (see  Figure  3-1).  Four  of  the  Phase  I  demonstrators  selected  for  Phase  II  were  assigned  to  the 
16B-hectare  area  to  minimize  any  advantages  over  first-time  demonstrators.  The  quantities  and  types  of 
ordnance  emplaced  in  the  16B-hectare  area  were  similar  to  those  emplaced  in  the  16 A-hectare  area.  A  new 
target  map  was  created  reflecting  the  changes  made  (PRC  1995a). 
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3 2  DEMONSTRATOR  SELECTION  PROCESS 

For  Phase  II  of  the  UXO  ATD  program,  potential  candidates  were  identified  through  a  Commerce  Business 
Daily  solicitation  (CBD  1994).  Interested  parties  were  then  sent  information  packages  that  included 
information  about  the  site  along  with  a  description  of  the  criteria  for  selection.  A  total  of  42  proposals  were 
submitted  for  Phase  II  of  the  UXO  ATD  program. 

Criteria  for  proposal  consideration  included  the  following:  corporate  experience,  key  personnel,  system  and 
technology  description,  support  and  quality  control,  ability  to  meet  the  government’s  requirements,  and  cost. 
Rased  (m  rhftw  criteria,  the  government  review  panel  selected  18  demonstrators.  Two  vendors  canceled  and 
one  government  laboratory  (Wright  Laboratory)  was  added  for  a  total  of  17  demonstrators  for  Phase  II. 

33  DEMONSTRATION  PROCEDURES 

All  demonstrators  chosen  were  provided  with  a  demonstrati  on  work  plan  (DWP)  that  outlined  operations  and 
safety  procedures  and  responsibilities  for  the  parties  involved  in  the  demonstrations.  Specifically,  the  DWP 
provided  site  background,  demonstration  responsibilities,  evaluation  criteria,  and  data  validation  information. 
A  data  entry  disk  was  provided  to  each  demonstrator  to  ensure  standard  data  submission  to  the  government 
for  evaluation.  Demonstrators  were  also  provided  with  copies  of  various  forms  used  by  PRC  during 
oversight  of  the  activities.  In  addition,  the  Safety,  Health,  and  Emergency  Response  Plan  (SHERP)  was 
included  in  the  DWP  for  use  by  demonstrators  (PRC  1995b). 

The  SHERP  served  as  a  guide  for  all  government  representatives,  support  personnel,  and  demonstrators.  At 
the  beginning  of  each  demonstration  period,  all  demonstration  team  members  were  required  to  attend  a 
comprehensive  health,  safety,  and  operations  briefing.  As  part  of  the  daily  routine,  each  demonstration  team 
member  was  required  to  acknowledge  their  compliance  with  the  SHERP  by  signature  before  field  activities 
began.  Each  demonstration  area  was  assigned  a  site  safety  officer  who  was  responsible  for  SHERP 
ramplianr.?;  An  EOD  technician  was  part  of  the  support  team  to  address  any  ordnance  issues. 

Demonstrators  were  provided  with  daily  weather  forecasts,  including  data  collected  from  the  on-site  weather 
cfatinn  and  reports  from  U.S.  Weather  Bureaus  in  Indianapolis,  Indiana,  and  Louisville,  Kentucky.  Forecasts 
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included  the  daily  high  and  low  temperatures,  relative  humidity,  wind  speed  and  direction,  chance  of 
precipitation,  and  likelihood  of  severe  weather. 

Demonstrators  were  given  a  specified  amount  of  time  to  collect  data,  based  on  the  type  of  technology  to  be 
demonstrated.  Detection  technology  demonstrators  with  ground  systems  were  given  40  hours  to  cover  the 
assigned  16-hectare  area.  Demonstrators  were  not  permitted  to  remove  any  objects  from  the  site  during  the 
demonstration. 

Detection  technology  demonstrators  with  airborne  systems  had  24  hours  to  characterize  the  32-hectare  area. 
Specific  air  routes,  flight  plans,  ground  tracks,  and  air  speeds  were  established  for  the  airborne 
demonstrations  to  avoid  conflicts  with  ongoing  Indiana  Air  National  Guard  training. 

Remediation  technology  demonstrators  had  24  hours  to  demonstrate  the  system’s  capability.  Demonstrators 
were  provided  with  baseline  targets  to  remediate. 

3.4  QUALITY  ASSURANCE  AND  QUALITY  CONTROL 

The  quality  assurance  program  plan  (QAPP)  for  Phase  II  outlined  quality  assurance  (QA)  and  quality  control 
(QC)  procedures  for  the  UXO  ATD  program.  The  QAPP  defined  rigorous  standard  procedures  for 
conducting  a  controlled  test  program.  The  primary  focus  of  the  QAPP  was  on  the  accuracy  of  the  emplaced 
baseline  target  set  (inert  ordnance  type,  location,  and  orientation),  target  validation,  the  transmission  of  data 
from  the  demonstrator  to  the  govemment,and  the  validation  of  algorithms  to  assess  system  performance. 

3.4.1  Target  Emplacement 

All  inert  ordnance  and  nonordnance  were  serialized  and  emplaced  in  accordance  with  a  site  layout  plan.  The 
location  of  all  baseline  targets  were  accurately  determined  by  a  licensed  surveyor  to  an  accuracy  of 
5  centimeters  in  three  dimensional  space.  All  maps  were  controlled  by  the  Government  to  ensure  site 
integrity. 
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3.4.2 


Target  Validation 


As  part  of  the  QA  procedures,  remediation  demonstrators  were  directed  to  specific  targets.  Locations  where 
ordnance  was  excavated  were  recorded.  In  this  way,  the  remediation  demonstrations  also  served  the  purpose 
of  target  validation.  As  each  target  was  unearthed,  PRC  personnel  verified  and  recorded  the  location,  depth, 
ci7P  cjasSj  and  serial  number  of  the  target  No  discrepancies  were  found  to  exceed  the  QAPP  requirements. 

3.4.3  .  Data  Transmittal  Control 

Chain-of-custody  procedures  were  implemented  to  ensure  accurate  data  transmittal.  All  data  not  transmitted 
in  person  were  transmitted  using  a  courier  service  equipped  with  a  unique  tracking  system  for  shipment  items. 
This  approach  allowed  data  transmittals  to  be  tracked  between  parties. 

3.4.4  Data  Submittal  and  Algorithm  Validation 

Two  r/wnpnter  software  programs  ware  developed  for  use  during  the  demonstrations:  the  data  entry  program 
and  the  target  matching  algorithm  (TMA).  The  data  entry  program  provided  a  means  for  the  demonstrator  to 
electronically  record  data  at  the  controlled  site  and  then  provide  the  data  for  analysis  in  a  standard  data 
submission  format  The  data  entry  disk  included  demonstration  condition  data,  target  baseline  data, 
demonstrator  target  data,  validated  target  data,  site  operational  and  logistical  data,  and  demonstrator 
descriptive  data.  The  data  entry  program  enabled  the  demonstrators  to  collect  and  report  their  target  and 
descriptive  data  consistently  and  reliably.  Subsequent  to  demonstrator  submittal  of  data,  a  copy  of  the  data 
was  returned  to  each  demonstrator  for  verification  of  data  accuracy .  If  necessary ,  any  discrepancies  or 
inconsistencies  were  resolved. 

The  TMA  was  designed  to  measure  demonstrator  effectiveness  in  locating  known  (baseline)  targets  at  the 
controlled  site  demonstration  areas.  During  the  operation  of  the  algorithm,  each  demonstrator's  target 
predictions  are  compared  to  baseline  target  positional  data  using  the  x,  y,  and  z  coordinate  system.  The 
Institute  for  Defense  Analysis  (IDA)  and  PRC  selected  the  algorithms  to  be  used  and  validated  the  computer 
code  used  to  assess  demonstrator  performance.  Before  the  TMA  and  the  computer  code  were  used  to  assess 
demonstrator  performance,  they  were  validated  internally  by  PRC  and  externally  by  IDA.  A  more  thorough 
explanation  of  the  TMA  and  its  application  is  provided  below  in  Section  4.0. 
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4.0  PERFORMANCE  EVALUATION  METHODOLOGY 


The  techniques,  criteria,  and  methods  used  to  evaluate  individual  detection  demonstrator  performance  for 
Phase  II  of  the  UXO  ATD  program  at  JPG  arc  based  in  large  part  on  those  developed  for  the  UXO  ATD 
Phase  I  analysis  by  Automated  Research  Systems  Ltd.  (USAEC  1994)  and  the  Institute  for  Defense  Analysis 
(IDA  1995;  USAEC  1995).  The  remediation  evaluation  methodology  is  discussed  with  die  respective  results 
in  Section  5.0. 


4.1  APPROACH 

The  effectiveness  of  UXO  characterization  systems,  such  as  magnetometers,  electromagnetic  induction 
sensors,  or  ground  penetrating  radar,  depends  on  their  performance  in  three  general  categories! 

(1)  detection  -  the  ability  to  detect  targets;  (2)  localization  -  the  ability  to  accurately  determine  the  detected 
target’s  location  in  three-dimensional  space;  and  (3)  identification  and  classification  —  die  ability  to 
discriminate  between  UXO  and  non-UXO  targets  and  to  determine  the  characteristics  of  targets  identified  as 
UXO.  These  performance  categories  used  for  Phase  II  follow  the  evaluation  criteria  used  to  compare 
demonstrator  performance  during  Phase  I  (USAEC  1995).  Area  coverage  and  cost  information,  are  provided 
as  a  historical  record  and  are  not  used  to  assess  system  performance. 

Demonstrators  were  evaluated  against  the  baseline  target  set  for  the  areas  of  the  site  they  surveyed.  The  JPG 
16 A,  16B,  and  32  hectare  sites  are  divided  into  grid  cells  that  are  30.5-  by  30.5-  meters  (100-  by  100-  feet) 
square.  If  a  demonstrator  surveyed  less  than  50  percent  of  a  particular  grid  cell,  those  items  from  the  baseline 
target  set  along  with  any  of  that  demonstrator’s  reported  targets  were  excluded  from  the  performance 
assessment 

Remnants  of  a  metal  fence  were  detected  by  demonstrators  at  each  test  area.  All  demonstrator  reported 
targets  and  baseline  items  contained  within  an  8-meter  wide  (26  feet)  swath,  which  encompassed  each  fence 
segment,  were  excluded  from  the  performance  analysis.  Figures  4-1  and  4-2  indicate  fence  segment  locations 
on  the  16A-  and  16B-hectare  areas,  respectively,  along  with  reported  target  locations  from  one  of  the 
demonstrators.  Appendix  B  describes  the  algorithm  used  to  remove  the  reported  targets  and  baseline  items 
and  the  rationale  for  so  doing. 
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After  identifying  the  portion  of  the  baseline  target  set  based  on  the  area  covered  by  the  demonstrator  and 
removing  the  data  around  the  fence  lines,  TMAs  were  applied  to  demonstrator  reported  targets  to  determine 
which  demonstrator  reported  targets  were  scored  as  detections  of  baseline  items  and  •which  were  not  These 
algorithms  are  briefly  described  in  Section  4.1.1  below  and  in  greater  detail  in  Appendix  A.  After  scoring  the 
demonstrator  repeated  targets,  demonstrator  performance  was  determined  in  each  of  the  three  categories. 

This  performance  analysis  and  associated  statistical  measures  are  presented  in  the  following  subsections. 

4.1.1  Target  Matching  Algorithms 

Two  of  these  TMAs,  "CLOSEST”  and  "GROUP,  ”  were  selected  for  use  in  the  Phase  II  analysis  (USAEC 
1995  and  IDA  1995).  These  algorithms  were  developed  and  validated  by  IDA  during  Phase  I  of  the  UXO 
ATD  analysis  (USAEC  1994, 1995;  and  IDA  1995).  These  algorithms  are  believed  to  provide  the  best 
representation  of  demonstrator  ability. 

For  the  CLOSEST  TMA,  a  circle  of  the  radius  or  the  critical  radius  (R**)  is  centered  about  the  approximate 
geometric  center  location  of  each  of  the  baseline  target  items  in  the  x-y  horizontal  plane.  The  closest 
demonstrator  reported  target  falling  within  this  circle  is  matched  to  that  baseline  item.  If  a  reported  target  can 
be  matched  to  multiple  baseline  items,  a  tie-breaking  scheme  is  employed  that  minimizes  the  average  distance 
to  the  items  while  maximizing  the  number  of  matches.  If  no  reported  targets  are  within  this  circle,  the 
baseline  item  is  scored  as  undetected.  As  in  Phase  I,  the  Phase  DR^  was  designated  as  2  meters  for  ground 
demonstrators  and  5  meters  for  air  demonstrators. 

At  the  JPG  survey  sites,  several  of  the  baseline  items  were  emplaced  in  close  proximity  to  other  baseline 
items  to  assess  the  resolution  capabilities  of  the  various  systems.  Figures  4-3  and  4-4  are  cumulative 
distributions  of  the  relative  distances  between  baseline  items  and  their  closest  neighbor  at  the  16A-  and 
16B-hectare  areas,  respectively.  As  shown  in  these  figures,  between  37  and  38  percent  of  the  baseline  items 
are  within  2  meters  of  each  other.  Because  of  this,  demonstrators  with  larger  spatial  resolutions  may  have 
problems  resolving  individual  baseline  item  within  a  group  of  closely-spaced  items  and  instead  may  have 
reported  target  groups  as  a  single  target  Use  of  CLOSEST  TMA  provides  a  measure  of  the  system  s 
capability  to  detect  individual  UXO  items  in  an  area  highly  populated  with  UXO  and  debris. 
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The  GROUP  TMA  was  developed  for  use  in  determining  detection  performance,  because  it  is  believed  to 
provide  the  best  estimate  of  demonstrator  performance  in  this  category  (USAEC  1995;  IDA  1995).  Each 
group  essentially  mntaing  all  baseline  targets,  both  ordnance  and  nonordnance,  within  a  circle  of  a  2-meter 
radius.  For  this  TMA,  the  baseline  target  set  was  divided  into  groups  which  may  consist  of  one  or  more 
baseline  targets.  A  demonstrator  is  scored  as  detecting  a  group  if  his  reported  target  is  within  of  at  least 

cme  baseline  item  in  the  group.  If  no  reported  targets  are  within  of  a  group,  that  group  of  targets  is  scored 

as  undetected  Assuming  that  closely-spaced  baseline  items  are  not  detected  due  to  lack  of  resolution  in 
demonstrator  equipment,  the  GROUP  TMA  provides  the  best  estimate  of  demonstrator  detection 
performance.  Again  R^  was  2  meters  for  ground-based  demonstrators  and  5  meters  for  airborne 
demonstrators.  Any  demonstrator  reported  target  not  matched  to  a  baseline  ordnance  item  was  scored  as  a 
false  alarm. 

For  this  performance  assessment,  the  GROUP  TMA  was  used  to  quantify  demonstrator  detection 
performance.  A  different  TMA  was  required  for  localization,  identification,  and  classification,  because  the 
GROUP  TMA  combines  ordnance,  nonordnance,  and  items  of  different  size,  location,  and  depth,  into  a  single 
group.  As  a  result,  the  CLOSEST  TMA  is  used  for  localization,  identification,  and  classification,  because  it 
provides  a  one-to-one  matching  between  demonstrator  reported  targets  and  baseline  targets. 

4.1.2  Detection  Performance 

In  determining  the  detection  performance  of  any  system,  one  must  consider  how  that  system  reacts  to  both 
signal  and  noise.  For  example,  a  system  with  a  high  probability  of  detection  may  not  be  of  practical  use  if  the 
number  of  false  alarms  is  excessive,  because  false  alarms  require  investigation  and  unnecessary  expenditure 
of  resources. 

The  standard  method  of  comparing  the  performance  between  systems  employs  the  Receiver  Operating 
Characteristics  (ROC)  curve.  The  ROC  curve  is  a  plot  of  the  probability  of  detection  (PD)  of  a  detection 
system  as  a  function  of  the  probability  of  false  alarm  (PFA),  or  alternatively  the  false  alarm  rate  (FAR).  To 
produce  a  ROC  curve,  a  detection  threshold  is  applied  to  the  detection  system’s  output,  and  the  corresponding 
PD  and  PFA  are  determined  for  that  threshold.  The  threshold  level  is  varied  across  some  range  of  values,  and 
the  corresponding  P0  and  PFA  values  are  accumulated  and  plotted.  Figure  4-5  shows  a  sample  ROC  plot  for 
two  systems,  and  their  underlying  signal  and  noise  distributions  are  shown  in  Figures  4-6  and  4-7.  For  this 
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sample  plot,  both  the  signal  and  noise  distributions  are  assumed  to  be  Gaussian  with  unity  standard 
deviations  and  mean  signal  to  noise  ratios  of  2  and  3,  respectively. 

For  a  given  detection  threshold  in  Figures  4-6  and  4-7,  the  number  of  targets  detected  is  related  to  the  area 
under  the  signal  distribution  curve  that  is  to  the  right  of  the  threshold  level.  Likewise,  the  number  of  false 
alarms  is  related  to  the  area  under  the  noise  distribution  curve  that  is  to  the  right  of  the  threshold  level.  As 
seen  from  these  curves,  both  the  PD  and  PFA  increase  or  decrease  with  a  corresponding  decrease  or  increase  in 
the  detection  threshold,  as  reflected  in  the  ROC  curve  (see  Figure  4-5). 

When  the  individual  demonstrators  were  scored  against  the  baseline  target  set  for  JPG,  a  single  PD  and  PFA 
(and  FAR  )  value  was  obtained.  These  values  represent  a  single  point  on  the  ROC  curve  for  that  system.  If 
the  type  and  standard  deviation  of  both  the  signal  and  noise  distributions  were  known,  then  a  ROC  curve 
could  be  produced  for  that  system.  Because  the  characteristics  of  these  distributions  are  not  known,  ROC 
curves  were  not  produced. 

With  regard  to  signals,  probabilities  of  detection  are  computed  from  reported  targets  that  match  baseline 
items.  As  discussed  in  Section  4.1.1,  the  GROUP  TMA  was  used  to  match  demonstrator  reported  targets  to 
baseline  items  to  produce  the  primary  detection  performance  statistics.  Within  this  area,  probabilities  of 
detection  are  computed  for  the  ordnance,  nonordnance,  and  total  number  of  baseline  items  detected  versus  the 
corresponding  number  of  baseline  items  in  the  area  surveyed.  These  detection  probabilities  are  defined  as 
follows: 


PDori  =  Probability  of  detection  for  ordnance 

=  (number  ordnance  detected)/(total  number  ordnance  in  area  surveyed) 

Pr  , . =  Probability  of  detection  for  nonordnance 

=  (number  nonordnance  detected)/(total  number  nonordnance  in  area  surveyed) 

PD  =  Probability  of  detection  for  total  baseline  items 

=  (number  baseline  items  detected)/(total  number  baseline  items  in  area  surveyed) 

The  methods  described  above  for  determining  probability  of  detection  (PD)  may  include  a  percentage  of 
targets  detected  due  to  random  sensor  or  environmental  noise.  In  other  words,  correct  identification  of  an 
emplaced  item  could  result  from  a  random  point  For  example,  it  is  likely  that  a  certain  percentage  of  these 
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targets  would  fall  within  of  some  of  the  baseline  items  (Feller  1968).  Therefore,  a  detection  probability 

(labeled  in  the  tables  as  P  ■  -1  was  calculated  for  each  demonstrator.  is  a  measure  of  the  number  of 
baseline  items  that  would  be  detected  if  the  total  number  of  reported  targets  were  randomly  distributed  over 
die  surveyed  area,  as  opposed  to  being  specifically  declared  by  demonstrators.  This  number  is  useful  to 
compare  with  the  various  probabilities  computed  above  to  determine  if  the  PD  values  are  statistically  different 
from  the  detection  probability  that  would  result  from  a  random  placement  of  points.  If  PiaMkn  is  close  to  or 
greater  foan  a  demonstrator’s  P^,  it  is  likely  that  some  portion  of  those  detections  are  due  to  random  target 
declarations  P,  •  is  discussed  in  detail  in  Appendix  C  and  is  calculated  as  follows: 

Pran**,  =  l-e'X 


where 

A  =  np 

n  =  Number  of  demonstrator  reports 
p  =  Probability  of  having  a  report  within 


A 

A  =  Area  surveyed 


The  previous  ste*igrie<:  are  measures  of  demonstrator  ability  to  detect  ordnance  and  nonordnance.  System 
performance  must  also  include  determination  of  the  false  alarm  rate.  In  a  true  performance  assessment,  one 
must  cnngidar  the  false  alarm  rate  as  well.  By  lowering  the  detection  or  sensor  threshold,  one  can  increase  the 
PD  but  not  without  a  corresponding  increase  in  the  number  of  false  alarms.  This  increase  in  false  alarms 
ultimately  increases  the  cost  of  the  remediation  effort  Accordingly,  the  number  of  false  alarms  reported  by  a 
demonstrator,  the  corresponding  FAR,  and  the  probability  of  false  alarms  are  reported. 

During  Phase  I,  a  false  alarm  was  defined  as  a  reported  target  not  associated  with  a  baseline  target  item. 
However,  for  Phase  H,  the  remediation  efforts  on  the  16B-acre  area  showed  that  the  field  contained  a  large 
number  of  nonordnance  items  (see  Appendix  D).  Therefore,  this  analysis  uses  a  narrower  definition  of  false 
alarm  that  is  more  consistent  with  the  goals  of  a  remediation  system.  To  correspond  with  ordnance 
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remediation,  a  false  alarm  is  defined  as  a  demonstrator  reported  target  not  associated  with  ordnance.  To 

the  number  of  false  alarms,  demonstrator  reported  targets  not  matched  to  the  baseline  ordnance  set 
are  declared  false  alarms.  Because  only  a  small  number  of  nonbaseline  ordnance  items  were  remediated  in  the 
16B-hectare  area,  and  they  were  all  located  near  a  firing  range,  the  number  of  nonbaseline  ordnance  items  in 
the  controlled  sites  is  believed  to  be  negligible.  Similarly,  the  FAR  is  computed  by  dividing  the  number  of 
f?ig/»  alarm*;  NFA,  by  the  area  surveyed,  A  (in  square  meters): 


Nfa 

far  =  -%■ 

A 


The  probability  of  false  alarm,  PFA,  is  defined  as  the  fraction  of  area  in  the  site  covered  by  the  false  alarms, 
which  is: 


?fa  ~ 


NfakR, 


'Crit 


One  final  measure,  false  alarm  ratio,  assesses  the  impact  of  a  system’s  false  alarms  on  a  remediation  effort. 
The  false  alarm  ratio  is  defined  as: 


False  Alarm  Ratio 


_ Number.  of  False  Alarms • _ 

Number  of  Ordnance  Items  Detected 


The  statistic  defines  the  number  of  nonordnance  holes  that  would  be  excavated  for  every  hole  containing  an 
ordnance  item.  This  assumes  that  all  demonstrator  reported  targets  would  be  investigated  and  sufficient 
resources  would  be  available  to  support  this  excavation. 

4.13  Localization  Performance 

Localization  is  a  measure  of  how  accurately  demonstrators  can  determine  the  location  of  buried  ordnance  in 
three-dimensional  space.  Once  an  object  has  been  detected,  its  position  must  be  determined  horizontally 
(x,  y)  and  vertically  (z,  depth). 

For  this  analysis,  the  location  errors  for  each  demonstrator  were  computed  from  the  set  of  baseline  ordnance 
items  that  were  determined  using  the  CLOSEST  TMA.  As  a  result,  horizontal  location  errors  will  be 
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constrained  by  the  (2  meters  or  5  meters)  used  in  die  detection  process.  For  each  detection,  the  error  of 

the  reported  target  relative  to  the  approximate  geometric  center  of  the  baseline  ordnance  item  is  computed  for 
die  horizontal  and  depth  components  as  follows: 

dx  =  xr-xb 

4y  s  yr-yb 

dz  =  z-zb 

where  the  V  subscript  refers  to  the  demonstrator  reported  target  position  and  the  “b”  subscript  refers  to  the 
baseline  target  position.  A  negative  value  in  dx,  dy,  and  dz  indicate  that  the  reported  target  position  is  to  the 
west  and  south  of  and  shallower  than,  the  baseline  target 

The  errors  associated  with  determining  position  and  depth  can  have  two  components:  (1)  constant  offset 
(5r,  ~dy,  dz),  and  (2)  randomly  fluctuating  error  ( az).  These  components  of  die  error  are  estimated 
by  the  means  and  standard  deviations  of  dx,  dy,  and  dz,  respectively.  The  radial  distance  error,  r  (in  the 
horizontal  plane  only),  and  the  mean  absolute  depth  error  |dz|,  are  measures  of  the  average  total  error  in  the 
location  estimates  that  result  from  the  combination  of  the  offset  and  randomly  fluctuating  error  components. 
These  statistics  are  computed  as  follows  (where  “N”  represents  the  number  of  baseline  ordnance  targets 
detected): 

•  The  mean  and  standard  deviation  of  the  x  and  y  errors  of  the  reported  target  locations 
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The  m^n  and  standard  deviation  of  the  radial  distance  error  for  the  UXO  from  the  center  of 
mass 


Mean: 


Standard  Deviation:  or 


N 


N 


r,.  =  Jdx?+dy? 


•  The  mean,  standard  deviation,  and  mean  absolute  depth  errors  of  the  UXO 


Mean:  dz  = 


i a 

N 


Standard  Deviation:  oz 


Y  ( dzi  -  dz)2 


N 


N 


Mean  Absolute:  \dz\ 


N 


N 


The  **N”  in  the  denominator  of  the  above  equations  represents  the  number  of  baseline  ordnance  targets 
detected. 


4.1.4  Identification  and  Oassification  Performance 

In  any  remediation  system,  the  ability  to  correctly  identify  and  classify  a  detected  target  provides  valuable 
information  to  the  remediation  process.  Because  large  amounts  of  both  natural  and  man-made  nonordnance 
are  encountered  on  most  ranges,  the  ability  to  discriminate  between  ordnance  and  nonordnance  items  (referred 
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to  as  identification  in  this  analysis)  is  very  important  in  reducing  the  number  of  nonordnance  items  (false 
alarms)  that  are  detected.  In  addition,  estimates  of  ordnance  size  and  class  (such  as  bomb,  projectile,  mortar, 
or  cluster)  provide  guidance  on  which  precautions  should  be  taken  during  the  remediation  process. 

Phase  II  demonstrators  were  requested  to  identify  and  classify  reported  targets  by  type  (ordnance  or 
nonordnance);  size  (small,  medium,  or  large),  and  classification  (bomb,  projectile,  mortar,  or  cluster).  To 
assess  demonstrator  identification  and  classification  performance,  the  following  statistics  were  computed 
from  detected  baseline  items  using  the  CLOSEST  TMA : 


Pco  =  Probability  of  correct  identification  of  ordnance 

=  (number  ordnance  correctly  classified)/(number  ordnance  detected) 

Pq,  =  Probability  of  correct  identification  of  nonordnance 

=  (number  nonordnance  correctly  classified)/ (number  nonordnance  detected) 

Pcs  =  Probability  of  correct  size  determination  of  small  ordnance 

=  (number  small  ordnance  correctly  classified)/ (number  small  ordnance  detected) 

Pa*  =  Probability  of  correct  size  determination  of  medium  ordnance 

=  (number  medium  ordnance  correctly  classified)/(number  medium  ordnance  detected) 

Pcl  =  Probability  of  correct  size  determination  of  large  ordnance 

=  (number  large  ordnance  correctly  classified)/ (number  large  ordnance  detected) 

Peg  =  Probability  of  correct  classification  determination  of  bombs 
=  (number  bombs  correctly  classified)/ (number  bombs  detected) 

Pcp  =  Probability  of  correct  classification  determination  of  projectiles 

=  (number  projectiles  correctly  classified)/(number  projectiles  detected) 

Pcmo  =  Probability  of  correct  classification  determination  of  mortars 
=  (number  mortars  correctly  classified)/(number  mortars  detected) 

Pq.  =  Probability  of  correct  classification  determination  of  clusters 
=  (number  clusters  correctly  classified)/ (number  clusters  detected) 

These  identification  and  classification  probabilities  are  not  to  be  confused  with  the  classification  ratios 
presented  in  the  Phase  I  report  (USAEC  1994).  The  Phase  II  probabilities  measure  a  demonstrator’s  ability 
to  correctly  identify  and  classify  reported  targets,  while  Phase  I  classification  ratios  measured  the 
demonstrator’s  classification  ability  using  the  entire  baseline. 
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In  yMifmn  to  the  identification  and  classification  probabilities  by  category,  detection  probabilities  were  also 
computed  from  the  CLOSEST  TMA  as  follows: 


Poo  -  Probability  of  detecting  ordnance 

=  (number  ordnance  detected) / (number  ordnance  items  in  area  surveyed) 

Pm  -  Probability  of  detecting  nonordnance 

=  (number  nonordnance  detected) /(number  nonordnance  items  in  area  surveyed) 

PDS  =  Probability  of  detecting  small  ordnance 

=  (number  small  ordnance  detected)/ (number  small  ordnance  items  in  area  surveyed) 

PDMc  =  Probability  of  detecting  medium  ordnance 

=  (number  medium  ordnance  detected)/ (number  medium  ordnance  items  in  area  surveyed) 

Pdl  =  Probability  of  detecting  large  ordnance 

=  (number  large  ordnance  detected)/(number  large  ordnance  items  in  area  surveyed) 

PDB  =  Probability  of  detecting  bombs 

=  (number  bombs  detected) /(number  bomb  items  in  area  surveyed) 

Pdp  =  Probability  of  detecting  projectiles 

=  (number  projectiles  detected)/ (number  projectile  items  in  area  surveyed) 

PDMo  =  Probability  of  detecting  mortars 

=  (number  mortars  detected)/(number  mortar  items  in  area  surveyed) 

Poc  =  Probability  of  detecting  clusters 

=  (number  clusters  detected)/ (number  cluster  items  in  area  surveyed) 


42  PERFORMANCE  ASSESSMENT 

42.1  Performance  Assessment  Plots 

In  the  individual  demonstrator  summaries  (see  Section  5.0),  two  plot  types  are  presented  to  graphically 
display  individual  demonstrator  performance  abilities  and  to  allow  comparisons  among  them.  In  this  section, 
examples  of  these  plots  are  presented  and  discussed 

The  shape  and  magnitude  of  a  sensor  signal  output  are  functions  of  both  the  size  and  depth  (distance  from  die 
sensor)  of  the  buried  ordnance  item.  For  magnetometer  systems,  the  “size”  component  is  a  complicated 
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function  of  ferrous  mass,  diameter,  length,  and  orientation.  For  electromagnetic  induction  and  GPR  sensors, 
the  “size”  is  effectively  related  to  the  cross-sectional  area  of  the  buried  ordnance  item.  In  both  cases,  the  size 
rnmpnnmt  of  these  types  of  systems  depends  to  varying  degrees  on  the  diameter  of  the  ordnance  item. 

To  illustrate  this  dependency,  a  scatter  plot  of  size  (as  indicated  by  the  diameter  of  the  ordnance)  versus  depth 
of  the  baseline  ordnance  item,  was  produced  (see  Figures  4-8  and  4-9).  In  these  figures,  detected  ordnance 
items  (as  matched  by  the  CLOSEST  TUK,  with  the  area  searched  and  fence  line  removed)  are  indicated  by 

darkened  squares;  those  items  not  detected  are  indicated  by  open  squares.  The  regions  corresponding  to  the 

three  size  classes  are  indicated  by  dashed  lines.  To  avoid  biases  from  nearby  ordnance,  only  those  ordnance 
items  not  having  another  baseline  item  within  2.0  meters  of  its  horizontal  (x,  y)  location  were  plotted;  as  a 
result,  the  number  of  detected  and  undetected  ordnance  items  will  differ  from  the  numbers  generated  in 
Section  4.1.4.  Figures  4-8  and  4-9  illustrate  ordnance  detection  capabilities  with  respect  to  depth  and  size. 
These  figures  show  the  detection  limits  of  the  demonstrator  systems.  For  example,  in  Figure  4-8,  the 
demonstrator  is  successful  at  detecting  medium  and  large  targets  but  has  difficulty  detecting  small  targets  at 
most  depths.  This  figure  illustrates  a  relationship  for  ordnance  detection  with  respect  to  depth  and  size.  In 
Figure  4-9,  no  such  relationship  exists,  as  detections  appear  to  occur  at  random  with  respect  to  depth  and 

size. 


As  a  measure  ofaUXO  detection  system’s  effectiveness  as  it  relates  to  remediation,  the  PD  for  ordnance 
is  plotted  against  the  false  alarm  ratio  for  each  demonstrator  in  Figure  4-10.  A  comparison  of  these 
two  statistics  define  the  fraction  of  ordnance  items  that  would  be  remediated  and  the  number  of  nonordnance 
holes  that  would  be  excavated  for  every  hole  containing  an  ordnance  item.  The  GROUP  TMA  was  used  to 
generate  these  statistics  (over  the  area  surveyed  and  with  the  data  around  the  fence  line  removed).  This 
assumption  is  based  on  the  rationale  that,  in  a  remediation  effort,  all  reported  targets  would  have  to  be 
investigated.  These  statistics  represent  estimates  of  how  well  a  system  would  perform  if  sufficient  resources 
were  available  to  remediate  every  reported  target  location,  and  if  the  remediator  were  100  percent  effective. 

Good  performance  in  this  category  is  denoted  by  symbols  located  in  the  plot’s  upper  left  quadrant  The  plot 
shows  two  things:  (1)  a  relatively  high  number  of  baseline  ordnance  items  were  detected,  and  (2)  if 
remediation  was  performed  based  on  these  detection  data,  a  relatively  low  number  of  holes  excavated  would 
not  r/Mitain  ordnance.  Likewise,  poor  performance  in  this  category  is  denoted  by  symbols  located  in  the 
plot’s  lower  right  quadrant  Statistics  found  here  indicate  the  opposite:  (1)  a  relatively  low  number  of 
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baseline  ordnance  items  were  detected,  and  (2)  if  remediation  was  performed  based  on  these  detection  data,  a 
relatively  high  number  of  holes  excavated  would  not  contain  ordnance.  Simply  stated,  the  more  holes 
remediated  that  do  not  contain  ordnance,  the  more  costly  the  remediation  effort 

4 22  Confidence  Intervals 

Because  there  were  a  finite  number  of  baseline  targets  used  in  Phase  H,  the  performance  statistics  computed 
for  each  demonstrator  are  estimates  of  the  true  statistics  because  each  demonstrator’s  survey  during  Phase  II 
represents  a  single  sample  drawn  from  a  random  process.  Due  to  the  random  statistical  fluctuations,  the  true 
performance  lie  within  some  interval  about  this  estimate.  This  interval  is  referred  to  as  the 

confidence  interval  (Cl). 

of  these  fluctuations,  a  Cl  must  be  computed  for  the  probabilities  and  means  before  comparisons 
between  performance  statistics  czn  be  made.  The  computation  of  Cl  allows  one  to  determine  whether  or  not 
the  observed  differences  in  performance  are  statistically  significant  For  the  detection  and  classification 
probability  estimates,  the  Cl  is  computed  as  follows: 

CJ.  -  P  ±  2X  _af2 

where 

P  =  Probability  of  detection  or  classification 
N  =  Total  number  of  elements  in  the  test  set 
z\-an  ~  Standard  normal  deviate  at  the  1  -a/2  level 

For  estimates  of  the  mean,  the  Cl  is  computed  as  follows: 


CJ.  =  ]i  ±  z 


l-cc/2 


y/N 
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where 


|i  =  Fcrimafft  of  the  mean 
a  =  Estimate  of  the  standard  deviation 
N  =  Total  number  of  elements  in  the  test  set 
z  =  Standard  normal  deviate  at  the  1  -a/2  level 

-a/2 


For  a  95  percent  Cl,  Zj.^  ~  1-96. 


To  illustrate  how  Cl  is  used  in  the  probability  analysis  presented  in  this  report,  let  us  assume  that  there  are 
three  demonstrators  (A,  B,  and  C)  with  the  following  PD  values,  values,  and  number  of  baseline  targets 

(N)  in  the  area  they  surveyed: 


For  A: 

Pd  =  0.65, 

PnDdom  =  0.25, 

and  N  =  50 

ForB: 

Pd  =  0.35, 

P«d»  =  0.25, 

andN  =  50 

ForC: 

Pd  =  0.60, 

P™»iob  “  0.25, 

andN  =  5 

As  a  first  step,  we  must  determine  if  these  PD  values  from  each  demonstrator  are  significantly  different  than 
the  PD  values  that  would  be  obtained  by  randomly  distributing  their  reported  targets  over  the  site  (as  indicated 
by  their  P  •  ^  Using  the  above  formula,  the  95  percent  Cl  for  each  demonstrator’s  PD  value  can  be 
determined  as  follows: 


For  A:  Cl  =  0.13;  0.52  *PDs  0.78 
ForB:  Cl  =  0.13;  0.22  *PD£  0.48 
For  C:  Cl  =  0.42;  0.23  s  PD  s  1.00 

Based  on  these  numbers,  the  demonstrator  PD  values  are  significantly  different  than  their  P,***,  values  if  their 

respective  P . . . values  do  not  fall  within  the  Cl.  For  these  three  demonstrators,  only  demonstrator  A  has  a 

PD  value  that  is  significantly  different  (at  the  95  percent  level)  than  the  P^ndon  value. 
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Cumulative  Distribution 


Cumulative  Distribution  of  Distance  Between  Nearest  Neighbor  Baseline  Items  at  16A-Hectare  Area 
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Cumulative  Distribution 


Figure  4  4  _ 

Cumulative  Distribution  of  Distance  Between  Nearest  Neighbor  Baseline  Items  at  16B-Hectare  Area 
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Probability  Detection 


Figure  4-5 

Sample  Receiver  Operating  Characteristics  (ROC)  Curve 


- -  Mean  Signal-to-Noise:  2 

-  Mean  Signal-to-Noise:  3 
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Sample  Signal  and  Noise  Distribution  with  Mean  Signal  to  Noise  Ratio  (SNR)  =  2 


- -  Noise  Distribution 

-  Signal  Distribution 
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Probability  Density  Function 


Figure  4-7  _  . 

Sample  Signal  and  Noise  Distribution  with  Mean  Signal  to  Noise  Ratio  (SNR)  -  3 


Detection  Threshold 


—  _  .  Noise  Distribution 
-  Signal  Distribution 
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Depth  (meters) 


Sample  Plot  Illustrating  Detection  as  a  Function  of  Depth  and  Size  (Diameter)  Trend  Indicated 
(only  those  ordnance  items  not  having  another  baseline  item  within  2  meters  of  its  horizontal  (x,y)  location 

are  shown) 


Small  Medium  Large 


■  Target  Detected 
□  Target  Not  Detected 


SamplePtot  Illustrating  Detection  as  a  Function  of  Depth  and  Size^am^^^T«gfadica^ 
(only  those  ordnance  items  not  having  another  baseline  item  within  2  meters  of  its  horizontal  (x,y) 

are  shown) 


Small  Medium  Large 


Principal  Diameter  (millimeters) 


■  Target  Detected 
□  Target  Not  Detected 


Figure  4-10 

Plot  of  Demonstrator  Probability  of  Detection  (Ordnance)  versus  False  Alarm  Ratio 
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False  Alarm  Ratio  (False  Alarms/Ordnance  Items  Detected) 


5.0  DEMONSTRATION  RESULTS 


During  Phase  n,  15  UXO  detection  systems  and  two  remediation  systems  were  demonstrated  from  May 
through  September  1995.  Detection  system  technologies  included  eight  magnetometer  systems,  seven 
electromagnetic  induction  systems,  and  five  ground-penetrating  radar  (GPR)  systems;  in  addition,  five 
companies  utilized  a  multi-sensor  approach.  Of  the  detection  systems,  six  were  man-portable,  two  were 
vehicle-towed,  four  were  combined  man-portable  and  vehicle-towed  (multimodal),  and  three  were  airborne. 

5.1  MAGNETOMETER  SENSOR  SYSTEMS 

5.1.1  Aerodat  Inc. 

Aerodat  Inc.  (Aerodat),  demonstrated  from  September  6  through  8, 1995,  at  the  32-hectare  area  at  JPG. 
Aerodat  also  participated  in  Phase  I  of  the  UXO  ATD  program. 

5.1.1.1  Technology  Description 

Aerodat  used  a  magnetometer  system  for  its  airborne  demonstration  at  JPG  (see  Figure  5. 1 . 1  -1).  The 
magnetometers  are  mounted  on  a  “bird,”  which  is  towed  by  a  helicopter.  Aerodat’s  system  consists  of  five 
cesium  total-field  magnetometers  and  a  three-component  fluxgate  magnetometer,  all  with  a  resolution  of 
0.01  nanotesla.  Four  of  the  cesium  sensors  are  arranged  in  the  pods  of  a  rigid  X-wing  “bird,”  with  one  more 
in  the  nose,  and  the  fluxgate  in  the  tail.  Aerodat  states  that  this  configuration  provides  three-component  field 
and  gradient  measurements,  yielding  multiple  independent  readings  for  each  sample.  Data  are  sampled  every 
0.1  second,  which  averages  2  to  3  meters  (6  to  9  feet)  ground  spacing  at  typical  air  speeds  (Aerodat  1995a). 

After  acquisition,  data  reduction  occurs  as  heading  and  lag  corrections  are  applied  as  calculated  in  the  pre- 
survey  calibration  flights.  The  diurnal  correction  is  made  by  directly  subtracting  the  base  station 
measurements.  Global  positioning  satellite  (GPS)  data  are  differentially  corrected  and  recorded  but  can  be 
postprocessed  if  the  real-time  link  is  broken  for  any  reason.  The  GPS  antenna  is  mounted  directly  on  the 
bird  for  greater  accuracy  of  data  positioning  and  anomaly  targeting. 
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Figure  5.1.1-1  Aerodat  Inc..  Airborne  Detection  System 


The  sensors  and  recording  instruments  are  installed  in  the  helicopter  once  it  is  on  site.  Ground  stations  for 
diurnal  magnetics  and  differential  GPS  measurements  are  established  at  the  survey  site  for  maximum 
effectiveness.  A  low-power,  radio  modem  link  is  established  from  the  GPS  base  to  the  airborne  unit  to 
provide  real-time  updates  of  position.  According  to  Aerodat,  postprocessed  differential  GPS  is  adequate  for 
data  positioning,  but  in-flight  navigation  requires  real-time  updates  to  ensure  full  and  even  coverage  of  the 
area. 

Instruments  are  calibrated  off  site  after  installation.  This  takes  place  as  a  cloverleaf  pattern  is  flown  at  high 
altitudes  so  that  heading  corrections  and  absolute  sensor  differences  can  be  calculated.  A  low  pass  over  a 
large  and  obvious  target  is  also  flown  in  two  directions  to  calculate  the  recording  time  lag.  These  test  flights 
also  serve  to  verify  the  real  time  GPS  link.  Once  this  is  completed,  the  survey  is  conducted.  At  JPG,  the 
sensor  was  towed  across  the  target  area  in  two  orthogonal  directions,  using  preprogrammed  navigation 
points  from  the  32-hectare  area.  Lines  were  spaced  10  meters  (33  feet)  apart.  The  average  survey  altitude 
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was  25  to  30  meters  (82  to  98  feet).  The  altitude  is  monitored  by  a  laser  altimeter  mounted  in  the  bird,  and  a 
radar  altimeter  mounted  in  the  helicopter.  The  laser  altimeter  provides  a  more  accurate  measurement  of  the 
sensor  height  but  cannot  penetrate  foliage  and  tree  cover  as  the  radar  can.  The  radar  cannot  be  mounted  in 
the  helicopter,  because  it  disrupts  the  magnetic  readings.  The  orientation  of  the  bird  in  flight  is  measured  as 
a  by-product  of  the  fluxgate  magnetometer  data  collection  (Aerodat  1995a). 

For  the  demonstration,  data  were  transcribed  to  a  working  database  for  processing  and  presentation.  Data 
from  the  base  station  monitor  were  used  to  remove  the  effects  of  diurnal  variations  from  the  total  field  data. 
Vertical  and  horizontal  gradient  data  were  corrected  for  sensor  pitch  and  roll  using  an  on-board  fluxgate 
magnetometer. 

Aerodat  indicated  the  following  changes  from  Phase  I:  identification  of  false  anomalies  caused  by  sensor 
pitch-and-roll  motions;  redesign  of  the  “bird”  to  provide  a  three-component  field  and  gradient  sensor;  and 
data  processing  improvements  (Aerodat  1995a  and  b). 

5. 1.1.2  System  Assessment 

This  section  summarizes  the  demonstration  by  Aerodat,  based  on  observations  made  in  the  field  and 
information  provided  by  Aerodat. 

Requirements  for  Technology  Implementation 

Aerodat  used  five  people  to  complete  its  demonstration.  Two  people  were  in  the  aircraft,  one  processing  data 
and  the  other  flying  the  helicopter.  The  remaining  personnel  served  as  ground  crew.  All  equipment  other 
than  the  helicopter  was  transported  to  JPG  from  Aerodat’ s  office  in  Mississauga,  Ontario,  Canada. 

Support  equipment  required  to  perform  the  survey  included  a  Bell  206  helicopter,  equipped  with  the  system- 
required  instrumentation;  a  van  that  transported  the  system  equipment;  computers;  and  a  GPS  base  station. 
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Operational  Capabilities  and  Limitations 


The  survey  was  flown  completely  in  two  orthogonal  directions.  The  aircraft  ground  speed  was  maintained  at 
about  25  meters  per  second  (82  feet  per  second).  The  nominal  sensor  height  was  30  to  35  meters  (98  to 
115  feet)  to  ensure  the  safety  of  the  aircraft  and  crew.  Aerodat  stated  in  its  results  that  the  ideal  sensor 
height  for  this  system  is  less  than  1 0  meters  (33  feet)  and  never  more  than  15  meters  (49  feet).  This  was  not 
mentioned  as  a  limiting  factor  in  the  proposal.  Tree  cover  prevented  flying  any  lower  than  30  to  35  meters 
(98  to  1 15  fest)  throughout  the  area.  According  to  Aerodat,  this  survey  height  reduced  the  sensitivity  of  the 
system  to  4  percent.  No  delays  occurred  as  a  result  of  equipment  failure. 

5.1.1 3  Measured  Performance 

Aerodat  covered  the  entire  32-hectare  area  within  13  of  the  allotted  24  hours.  Aerodat  identified  a  total  of 
78  potential  UXO  locations  at  the  32-hectare  area.  Figure  5.1. 1-2  shows  Aerodat’s  target  declarations  with 
its  airborne  magnetometer  system  at  the  32-hectare  area  (Aerodat  1995b).  Aerodat  was  scored  on 
100  percent  of  the  total  area.  Aerodat’s  performance  is  shown  in  Table  5.1. 1-1,  which  presents  both 
detection  and  localization  statistics.  Aerodat  did  not  provide  type,  size,  and  class  information  so 
classification  statistics  were  not  computed. 

The  baseline  for  computing  detection  performance  included  both  ordnance  and  nonordnance  items.  The 
detection  ratio  of  0.02  (cited  as  PD  ordnance  in  Table  5.1.1-1)  reflects  the  number  of  targets  detected  by 
Aerodat  compared  to  the  total  number  of  baseline  ordnance  targets  in  the  area  covered.  This  detection 
probability  is  identical  to  the  probability  of  detection  arising  from  random  declarations  due  to  sensor  noise 
and  other  factors  (Prandom  =  0.02).  Aerodat’s  FAR  was  2.3  per  hectare. 

Aerodat  had  the  lowest  probability  of  detection  of  the  four  magnetometer  systems  demonstrated.  However, 
Aerodat’s  detection  results  are  consistent  with  those  obtained  by  other  airborne  demonstrators.  These  results 
may  be  due  to  the  difficulty  of  magnetometers  to  detect  at  the  high  speed  of  an  airborne  system.  Aerodat  s 
FAR  was  low,  which  may  be  indicative  of  a  combination  of  both  a  high  noise  level  and  a  high  detection 

threshold. 
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As  part  of  Phase  I,  Aerodat  had  a  PD  of  0.04  for  ordnance  items.  The  FAR  was  4.2  per  hectare.  Phase  II 
detection  performance  decreased  slightly  with  a  PD  of  0.02,  although  the  FAR  improved  slightly  as  it 
decreased  from  4.2  in  Phase  I  to  2.3  in  Phase  II.  Aerodat’ s  results  are  comparable  to  the  results  from  other 
airborne  systems  from  Phase  I  and  Phase  n. 

For  most  demonstrators,  the  probability  of  ordnance  detection  depends  on  both  the  size  and  depth  of  the 
buried  ordnance  item.  Figure  5. 1.1 -3,  which  is  a  scatter  plot  showing  detection  performance  as  a  function  of 
size  versus  depth,  illustrates  this  relationship  for  Aerodat.  In  this  figure,  no  clear  relationship  is  evident. 

The  majority  of  targets  were  difficult  for  Aerodat  to  detect. 

The  localization  error  statistics  section  of  Table  5. 1.1-1  indicates  Aerodat’s  ability  to  estimate  the  locations 
of  the  targets  declared.  Aerodat  reported  target  depths  between  0.10  and  0.30  meter  (0.33  and  0.98  foot) 
below  ground  surface. 
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TABLE  5.1. 1-1 

PERFORMANCE  STATISTICS  FOR  AERODAT 


Detection  Statistics 


Number  Baseline 

Number  Matched 

Pd* 

Ordnance 

201 

4 

0.02 

Nonordnance 

20 

1 

0.05 

Total 

221 

5 

0.02 

P 

0.02 

|  Number  False  Alarms 

74 

1  False  Alarm  Rate 

2.3/hectare 

|  False  Alarm  Ratio 

18.5 

II  Probability  False  Alarms 

0.0182 

Localization  Statistics 


Mean  (m)b 

Std.  Dev.c(m)  | 

H  Position  (x,y)  | 

dx 

-0.37 

2.05 

dy 

0.56 

2.15 

Radial 

2.29 

|  Depth  (z)  | 

1 _ ^ _ 

-1.87 

1.00 

! _ 

dzl 

2.07 

Identification  and  Classification  Statistics 


\  Number  Baseline  Number  Detected 

PDe  Number  Correct 

E2I 

Ordnance 

245 

4 

0 

0 

Nonordnance 

27 

1 

0.04 

0 

0 

_ _ _ 

Large 

41 

3 

0.07 

0 

Medium 

37 

1 

0.03 

0 

Small 

165 

0 

0 

0 

NAg  | 

(Class  | 

Bomb 

23 

2 

0.09 

0 

0 

Projectile 

73 

2 

0.03 

0 

0 

Mortar 

143 

0 

0 

0 

NA 

Cluster 

6 

0 

0 

0 

NA 

Notes:  “  Probability  of  detection  (based  on  Group  TMA,  fence  area  excluded) 

b  Meter 

c  Standard  deviation 

d  Square  root  of  the  mean  square  depth  error 

e  Probability  of  detection  (based  on  closest  TMA,  fence  area  excluded) 
r  Probability  of  correctly  classifying  (based  on  closest  TM  A) 

‘Not  applicable 
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Figure  5.1.1-3 
Aerodat  Detection  Ability 


Small  Medium  L 


■  Target  Detected 
□  Target  Not  Detected 


5.1.2 


Australian  Defence  Industries,  Pty.  Ltd. 


Australian  Defence  Industries,  Pty.  Ltd.  (ADI),  demonstrated  from  June  14  through  18,  1995,  at  the 
16B-hectare  area  at  JPG.  ADI  also  participated  in  Phase  I  of  the  UXO  ATD  program.  During  Phase  II,  ADI 
used  two  separate  man-portable  sensor  systems  for  its  demonstration:  magnetometer  and  electromagnetic 
induction.  The  following  sections  describe  ADI’s  magnetometer  technology,  assess  ADI’s  demonstration, 
and  analyze  ADI’s  magnetometer  results.  For  a  complete  discussion  on  ADI’s  electromagnetic  induction 
technology  and  analysis  of  both  the  magnetometer  and  electromagnetic  induction  data  combined,  see  Section 
5.4.1. 

5.1.2.1  Technology  Description 

ADI  used  a  TM-4  magnetometer  system  for  part  of  its  surveying  (see  Figure  5. 1 .2-1 ).  The  TM-4 
magnetometer  includes  an  imaging  magnetometer  with  integrated  processing  and  interpretation  software. 

The  TM-4  data  acquisition,  processing,  interpretation,  and  documentation  systems  are  designed  to  locate 


Figure  5.1 .2-1  Australian  Defence  Industries.  Pty.  Ltd.,  TM-4  Imaging 
Magnetometer 
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ferrous  sources.  The  unit  can  be  used  with  either  single  or  multiple  cesium-vapor  magnetometers.  The 
TM-4  magnetometer  measures  total  magnetic  field  and,  according  to  ADI,  is  capable  of  reading  the  total 
magnetic  field  to  a  sensitivity  of  0.01 1  nanotesla  at  a  rate  of  100  times  per  second.  Data  are  stored  in  solid- 
state  memory  and  transferred  to  a  personal  computer  (PC)  in  the  field.  The  data  are  then  gndded  for  unage 
processing  and  interpretation  (ADI  1995a). 

ADI  indicated  the  following  improvements  from  Phase  I:  increased  accuracy  to  the  odometer  of  the  TM-4 
magnetometer,  improved  positioning  system,  new  routines  for  data  processing  of  the  TM-4  magnetometer 
data,  and  the  addition  of  the  electromagnetic  induction-sensor  system  as  described  in  Section  5.4.1  (ADI 
1995a). 

5. 1.2.2  System  Assessment 

This  section  summarizes  the  ADI  magnetometer  demonstration,  based  on  observations  made  in  the  field. 
Requirements  for  Technology  Implementation 

ADI  used  seven  people  to  complete  its  demonstration.  Two  people  operated  the  TM-4  magnetometer  in  the 
field.  One  person  carried  the  sensors,  followed  by  another  person  who  carried  a  PC  tethered  to  the  sensors. 
The  remaining  five  people  operated  the  two  electromagnetic  induction-sensor  units,  alternating  to  allow  for 
rest  periods.  All  system  equipment  used  for  this  demonstration  was  shipped  to  JPG  from  ADI  headquarters 
in  Australia.  ADI  rented  a  utility  vehicle  locally  for  transportation.  ADI  used  the  support  trailer  to  store 
equipment  and  to  recharge  batteries. 

For  grid  navigation,  ADI  used  four  magnetic  marking  chains.  The  magnetic  chains  were  placed  along  east- 
west  grid  lines.  The  survey  lines  or  transects  were  conducted  in  a  north-south  fashion.  The  TM-4 
magnetometer  uses  a  cotton  string  odometer  to  track  its  position.  Each  time  the  TM-4  magnetometer  passed 
the  marking  chains,  that  particular  grid  was  “marked”  in  the  computer  memory. 


5-10 


Operational  Capabilities  and  Limitations 


The  TM-4  magnetometer  can  be  used  as  a  single,  dual,  or  quad-sensor  unit.  The  quad-sensor  configuration 
was  used  for  this  demonstration.  The  system  is  man-portable  and  is  subject  to  the  physical  limitations  of  the 
operator.  The  TM-4  magnetometer  weighs  10  kilograms  (22  pounds),  imposing  some  fitness  requirements 
on  the  operator.  The  TM-4  magnetometer  is  versatile  in  the  field.  For  example,  it  can  be  collapsed  to  a 
single  unit  for  surveying  inaccessible  areas  between  trees. 

5. 1.2. 3  Measured  Performance 

ADI  covered  the  entire  16B-hectare  area  with  the  TM-4  magnetometer  in  about  32  hours.  ADI’s  results 
were  analyzed  in  two  ways.  The  TM-4  magnetometer  data  were  analyzed  separately,  and  the  magnetometer 
and  electromagnetic  induction  data  were  analyzed  together,  for  insight  as  to  the  ferrous  and  the  nonferrous 
items  detected  (ADI  1995a).  Section  5.4.1  discusses  of  the  measured  performance  of  the  combined 
magnetometer  and  electromagnetic  induction  data  (ADI  1995b). 

After  removing  the  fence  line  area  from  the  area  searched,  ADI  reported  569  targets  with  the  TM-4 
magnetometer  (see  Figure  5.1 .2-2)  within  the  1 6  hectares.  ADI’s  performance  with  the  magnetometer  is 
shown  in  Table  5.1.2-1,  which  presents  the  following:  (1)  detection,  (2)  localization,  and  (3)  classification 
statistics  with  respect  to  type,  size,  and  class  for  ADI’s  magnetometer  data. 

The  baseline  for  computing  detection  performance  for  Phase  II  included  both  ordnance  and  nonordnance 
items.  The  detection  ratio  of  0.63  (cited  as  PD  ordnance  in  Table  5. 1.2-1)  reflects  the  number  of  ordnance 
targets  detected  by  ADI  with  the  TM-4  magnetometer.  This  PD  value  is  significantly  different  than  the 
probability  of  detection  resulting  from  random  declarations,  shown  as  0.04  (P^om)-  ADI  had  a  FAR  of 
3 1 .4  per  hectare  for  the  TM-4  magnetometer. 

As  part  of  Phase  I,  ADI  deployed  the  TM-4  magnetometer  for  data  acquisition  and  had  a  PD  of  0.48  for 
ordnance  items:  the  FAR  was  30.1  per  hectare.  A  comparison  of  Phase  I  and  Phase  II  detection  performance 
shows  that  ADI’s  detection  capability  improved  while  the  FAR  remained  essentially  the  same. 


5-11 


For  most  demonstrators,  the  probability  of  ordnance  detection  depends  on  both  the  size  and  the  depth  of  the 
buried  ordnance  item.  Figure  5.1 .2-3  presents  a  scatter  plot  showing  detection  performance  as  a  function  of 
size  versus  depth  to  illustrate  this  relationship.  ADI  was  more  successful  at  detecting  medium  and  large 
targets  than  smaller  targets.  Small  targets  (less  than  100 -mm  diameter)  were  difficult  for  ADI  to  detect  at 
most  depths. 

The  localization  error  statistics  section  of  Table  5. 1.2-1  indicates  ADI’s  ability  to  estimate  the  location  of  the 
targets  declared.  ADI  reported  target  depths  between  0.39  meter  (1.28  feet)  above  surface  to  6.08  meters 
(19.95  feet)  below  ground  surface.  ADI  provided  type,  size,  and  class  information,  as  shown  in  the  table. 


Figure  5.1 2-2 

ADI  Magnetometer  Target  Declarations 
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TABLE  5.1.2-1 

PERFORMANCE  STATISTICS  FOR  ADI  (MAGNETOMETER  DATA) 


Detection  Statistics 


Number  Baseline 

Number  Matched 

Pd‘ 

Ordnance 

92 

58 

0.63 

Nonordnance 

8 

6 

0.75 

Total 

100 

64 

0.64 

P 

0.04 

Number  False  Alarms 

515 

False  Alarm  Rate 

31.7/hectare 

False  Alarm  Ratio 

8.88 

Probability  False  Alarms 

0.0398 

Localization  Statistics 


|  Mean  (m)b 

Std.  Dev.c(m)  j| 

|  Position  (x,y)  \\ 

dx 

-0.05 

0.61 

dy 

-0.23 

0.56 

Radial 

0.74 

II  Depth  (z)  ! 

dz? 

0.14 

0.67 

Idz 

0.68 

Identification  and  Classification  Statistics 


l : 

Number  Baseline 

Number  Detected  |  PD*  |  Number  Correct  Pc  |j 

Ordnance 

125 

60 

0.48 

45 

0.75 

Nonordnance 

14 

9 

0.64 

1 

0.11 

Size  _ 

Large 

31 

20 

0.65 

13 

0.65 

Medium 

23 

14 

0.61 

8 

0.57 

Small 

69 

25 

0.36 

24 

0.96 

H  Class  il 

Bomb 

15 

13 

0.87 

10 

0.77 

Projectile 

51 

24 

0.47 

10 

0.42 

Mortar 

53 

20 

7 

0.35 

Cluster 

6 

3 

0.50 

0 

0 

Notes:  *  Probability  of  detection  (based  on  Group  TMA,  fence  area  excluded) 

b  Meter 

c  Standard  deviation 

d  Square  root  of  the  mean  square  depth  error 

*  Probability  of  detection  (based  on  closest  TMA  fence  area  excluded) 
'  Probability  of  correctly  classifying  (based  on  closest  TMA) 
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Depth  (meters) 


Figure  5.1 2-3 

ADI  Magnetometer  Detection  Ability 


Small  Medium  Large 


■  Target  Detected 
□  Target  Not  Detected 
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5.13 


Geometries,  Inc. 


Geometries,  Inc.  (Geometries),  demonstrated  from  July  26  through  30,  1995,  at  the  16A-hectare  area  at  JPG. 
Geometries  also  participated  in  Phase  I  of  the  UXO  ATD  program. 

5.13.1  Technology  Description 

Geometries  used  a  ground  based  multisensor  approach  to  complete  its  demonstration  at  JPG.  The  first 
system  used  was  the  hand-carried  G-858  magnetometer,  three  of  these  units  were  used  in  the  demonstration 
(see  Figure  5. 1.3-1).  The  G-858  consists  of  two  cesium  magnetometer  sensors  horizontally  separated  by 
0.76  meter  (2.5  feet)  and  located  about  0.46  meter  (1.5  feet)  above  the  ground.  Data  from  each  sensor  are 
recorded  individually  on  the  G-858,  along  with  operator  marked  fiducials  and  time.  Each  magnetometer 
contains  a  data  control,  acquisition,  and  field  display  unit.  Differential  GPS  and  along-track  markers  are 
used  to  provide  sub-meter  positioning  information.  In  addition,  each  operator  uses  an  audio  cross-track 
differential  GPS  steering  indicator  to  stay  on  line.  Each  operator  has  a  headphone  with  an  audio  tone  that 


Figure  5.13-1  Geometries,  Inc.  G-858  Magnetometer 
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has  a  different  frequency  to  indicate  left  and  right.  The  frequency  pulsing  rate  indicates  deviation  right  or 
left  of  the  survey  line  (Geometries  1995a). 


Before  conducting  the  survey,  the  area  to  be  covered  was  marked  with  green  survey  paint  at  30-meter 
(100-foot)  intervals  along  one  side  of  the  grid  cell  and  at  1.5  meter  (5.0-foot)  intervals  along  the  other  side  of 
the  grid  cell.  GPS  data  and  magnetic  sensor  data  was  continuously  transmitted  back  to  a  base  station  by  an 
RF  data  link.  The  data  were  then  processed  at  the  base  workstation  by  the  Arete  Engineering  Technologies 
Corporation  (AETC)  team,  subcontractor  to  Geometries.  The  data  was  processed  to  provide  targets  for  GDE 
Systems,  Inc.  (GDE),  subcontractor  to  Geometries;  the  GDE  team  used  GPS  coordinates  to  locate  the  targets. 
The  GDE  team  concentrated  on  potential  UXO  target  areas  identified  by  the  magnetic  data. 

The  GPR  system  used  an  all-terrain  vehicle  (ATV)  to  tow  a  trailer  that  carried  seven  broadband  antennas 
over  the  areas  identified  by  the  data  team  (see  Figure  5 . 1 .3-2).  The  ATV  also  carried  a  motor  generator;  a 
commercial  network  analyzer,  which  is  the  transmitter  and  receiver,  a  computer,  and  a  display.  The  GPR 
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system  operates  with  radio  waves  at  discrete  frequencies  between  1 80  and  720  megahertz  (MHz),  measuring 
near  field  reflectance  with  an  antennae  array  on  a  ground  vehicle.  Vehicle  motion  and  array  sampling 
combine  to  synthesize  a  data  swath,  and  a  computer  processes  the  data  into  images.  The  images  are  plan 
views  in  horizontal  sections,  which  are  orthogonal  to  the  vertical  sections  of  pulsed  GPR  (Geometries 
1995a). 

System  improvements  from  Phase  I  included  the  method  of  surveying  and  processing  data.  Differential  GPS 
was  added  for  navigation  across  grid  cells,  and  GPR  was  added  to  provide  a  ground-base  multisensor 
approach.  During  Phase  n,  Geometries  combined  the  efforts  of  AETC  and  GDE;  all  three  of  these 
companies  demonstrated  independently  during  Phase  I  (Geometries  1995a). 

5.U.2  System  Assessment 

This  section  summarizes  the  Geometries  demonstration,  based  on  observations  made  in  the  field. 

Requirements  for  Technology  Implementation 

Geometries  used  12  people  to  complete  its  demonstration.  The  three  magnetometers  were  operated  by 
two-person  teams.  One  person  carried  the  magnetometer,  while  the  other  person  recorded  the  line  numbers 
and  assisted  the  operator.  Personnel  on  these  teams  alternated  to  provide  rest  periods.  One  person  provided 
EOD  technical  support  for  the  demonstration.  AETC  provided  two  people  for  data  processing.  Three  people 
from  GDE  performed  the  GPR  survey. 

All  system  equipment  used  for  this  demonstration  was  shipped  or  brought  to  JPG  by  Geometries. 

Equipment  required  for  the  magnetometer  survey  included  three  G-858  units  in  dual-sensor  configuration, 
three  backpack-mounted  GPS  receivers  and  antennas,  three  data  radios,  three  backpack-mounted  12-volt 
batteries,  one  laptop  PC  for  data  downloading,  one  ATV  for  use  as  a  utility  vehicle,  one  magnetic  base 
station,  and  one  GPS  base  station.  The  GPR  system  required  the  following  components:  an  array  of  seven 
antennas,  a  receiver,  a  transmitter,  a  data  acquisition  computer,  and  an  ATV  for  towing  the  GPR.  The  on¬ 
site  data  processing  system  consisted  of  a  Sun  SPARK  workstation,  a  Windows  PC  terminal,  a  black-and- 
white  and  color  printer,  and  AETC  modeling  software. 
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Support  equipment  used  by  Geometries  included  several  vehicles  to  transport  personnel  and  supplies. 
Several  spare  magnetometer  sensors  were  shipped  by  overnight  delivery  to  replace  faulty  sensors.  The 
support  trailer  supplied  electricity  to  recharge  the  Geometries  system  batteries.  Data  analysis  also  took  place 
at  the  support  trailer. 

Operational  Capabilities  and  Limitations 

The  magnetometers  used  by  Geometries  required  little  maintenance  beyond  recharging  the  batteries. 
Geometries  experienced  a  delay  on  the  first  day  because  the  freight  forwarding  company  temporarily  lost  the 
crate  containing  the  backpacks,  GPS,  and  communication  equipment.  The  three  magnetometer  systems 
operated  simultaneously.  Each  team  surveyed  a  30-by-l  89-meter  (100-by-620-foot)  section  in  succession 
with  the  other  teams.  Sensor  problems  with  one  of  the  magnetometers  were  resolved  using  replacement 
sensors  shipped  by  overnight  delivery. 

According  to  Geometries,  the  magnetometers  collect  data  at  a  rate  of  10  magnetic  readings  per  second  with  a 
sensitivity  of  0.05  nanotesla.  Survey  speeds  were  initially  estimated  at  about  2  hectares  (5  acres)  per  hour 
for  each  of  the  three  systems.  The  extremely  hot  weather  and  delays  on  the  first  day  may  have  been  partially 
responsible  for  the  slower  survey  speed.  Heavy  brush  and  uneven  terrain  can  also  slow  the  survey  speed  due 
to  difficulty  with  maneuverability.  Geometries  stated  in  its  results  that  a  single-sensor  magnetometer  could 
have  penetrated  the  areas  of  thick  brush  better  than  the  dual-sensor  magnetometer.  The  GPR  system  was 
proposed  to  be  able  to  travel  at  least  10  kilometers  (6  miles)  per  hour.  Geometries  found  the  GPR  system  to 
be  severely  restricted  by  terrain  and  vegetative  cover.  Geometries  stated  in  its  results  that  there  is  a  need  for 
a  portable,  hand-carried  GPR  that  would  not  have  to  be  driven  or  dragged  over  the  surface. 

5.133  Measured  Performance 

Geometries  surveyed  the  entire  1 6A-hectare  area  with  its  magnetometer  system  in  the  allotted  40  hours  (see 
Figure  5.1 .3-3).  Geometries  was  unable  to  cover  all  of  the  area  with  its  GPR  system  and  unable  to  process 
all  of  the  data  collected.  As  a  result,  Geometries  requested  that  it  be  scored  on  its  magnetometer  data  alone 
(Geometries  1995b).  Geometries  reported  521  targets  within  the  16A-hectare  area.  Geometries’ 
performance  is  shown  in  Table  5.1.3-1,  which  presents  the  following:  (1)  detection,  (2)  localization,  and 
(3)  classification  statistics  with  respect  to  type,  size,  and  class. 
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The  baseline  for  computing  detection  performance  included  both  ordnance  and  nonordnance  items.  The 
detection  ratio  of  0.83  (cited  as  PD  ordnance  in  Table  5. 1.3-1)  reflects  the  number  of  targets  detected  by 
Geometries  compared  to  the  total  number  of  baseline  ordnance  targets  in  the  16  hectares  with  the  fence  line 
area  removed.  This  detection  probability  is  significantly  different  than  the  probability  of  detection  arising 
from  random  declarations  due  to  sensor  noise  and  other  factors  (Pr^dom  =  0.04).  Geometries  had  a  FAR  of 
25.2  per  hectare. 

Geometries  had  the  highest  PD  (0.83)  of  all  magnetometer  systems  demonstrated.  Geometries  teamed  with 
AETC  to  perform  the  data  processing  which  may,  in  part,  account  for  their  performance  relative  to  the  other 
systems  demonstrated. 

As  part  of  Phase  I,  Geometries  had  a  PD  of  0.21  for  ordnance  items.  The  FAR  was  7.5  per  hectare.  A 
comparison  of  Phase  I  and  Phase  II  detection  performance  shows  that  Geometries  obtained  much  higher  PD 
values  for  Phase  II,  0.83  as  compared  to  0.21 .  However,  Geometries  FAR  also  increased,  from  7.5  in  Phase  I 
to  26.7  during  Phase  II. 

For  most  demonstrators,  the  probability  of  ordnance  detection  depends  on  both  the  size  and  depth  of  the 
buried  ordnance  item.  Figure  5 . 1 .3-4,  which  is  a  scatter  plot  showing  detection  performance  as  a  function  of 
size  versus  depth,  illustrates  this  relationship  for  Geometries.  Geometries  was  successful  at  detecting  targets 
of  all  sizes  at  most  depths. 

The  localization  error  statistics  section  of  Table  5. 1.3-1  indicates  Geometries’  ability  to  estimate  the 
locations  of  the  targets  declared.  Geometries  reported  target  depths  between  0  and  5.84  meters  (0  and  19.16 
feet)  below  ground  surface.  Geometries  declared  all  target  detections  as  ordnance.  Geometries  did  provide 
size  and  class  information,  as  shown  in  the  table.  Geometries  did  not  correctly  classify  any  ordnance  items. 
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Northing  (meters) 


Figure  5.13-3 

Geometries  Target  Declarations 
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TABLE  5.1.3-1 

PERFORMANCE  STATISTICS  FOR  GEOMETRICS 


Detection  Statistics 


Number  Baseline 

Number  Matched 

Pd’  1 

Ordnance 

127 

105 

0.83 

Nonordnance 

41 

23 

0.56 

Total 

168 

128 

0.76 

P 

1  random 

0.04 

Number  False  Alarms 

416 

False  Alarm  Rate 

26.7/hectare 

False  Alarm  Ratio 

3.96 

1  Probability  False  Alarms 

0.0336 

Localization  Statistics 


i 

Mean  (m)b 

Std.  Dev.c(m)  j| 

H  Position  (x,y)  _ _ — - - 

dx 

-0.11 

0.48 

dy 

0.08 

0.60 

Radial 

0.65 

Depth  (z) 

d 2^ 

0.21 

0.59 

Idzl 

0.62 

Identification  and  Classification  Statistics 


1 - - 

Number  Baseline 

Number  Detected 

@1 

Number  Correct 

m 

BType 

Ordnance 

158 

99 

0.63 

99 

1.00 

Nonordnance 

65 

31 

0.48 

0 

0 

Size 

■■ 

Large 

35 

27 

0.77 

20 

BDr£K 

Medium 

53 

37 

0.70 

20 

0.54 

Small 

69 

35 

0.51 

28 

0.80 

Class 

Bomb 

21 

19 

0.90 

0 

0 

Projectile 

69 

45 

0.65 

0 

0 

Mortar 

59 

31 

0.53 

0 

0 

Cluster 

9 

4 

0.44 

0 

0 

Notes:  *  Probability  of  detection  (based  on  Group  TM A,  fence  area  excluded) 

b  Meter 

c  Standard  deviation 

d  Square  root  of  the  mean  square  depth  error 

e  Probability  of  detection  (based  on  closest  TM  A,  fence  area  excluded) 
f  Probability  of  correctly  classifying  (based  on  closest  TMA) 
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Depth  (meters) 


Figure  5.13-4 

Geometries  Detection  Ability 


Small  Medium  Large 


Principal  Diameter  (millimeters) 


■  Target  Detected 
□  Target  Not  Detected 
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5.1.4 


Polestar  Technologies,  Inc, 


Polestar  Technologies,  Inc.  (Polestar),  demonstrated  from  June  14  through  17,  1995,  at  the  16A-hectare  area 
at  JPG. 


5.1.4.1  Technology  Description 

Polestar’ s  magnetometer  system  used  a  magnetic  sensor  with  both  total  field  and  vector  gradiometer  readings 
(see  Figure  5. 1 .4-1).  The  magnetometer  is  designed  to  incorporated  “dual-mode”  detection  in  a  single 
sensor.  Navigation  was  achieved  through  a  precision  beacon  system  (PBS)  that  offered  0. 1  meter  position 
information,  even  in  vegetated  and  forested  conditions.  The  PBS  was  designed  to  prevent  the  failures  that 
can  occur  with  GPS  when  satellite  communication  is  disrupted  due  to  terrain,  structures,  or  vegetation. 

Polestar’s  magnetometer  system  uses  five  detection  heads  mounted  on  a  2.4-meter  (8.0-foot)  boom 
positioned  0.5  meters  (1.6  feet)  apart.  The  sensors  are  carried  by  two  people  using  a  backpack  harness,  one 
on  either  end  of  the  boom.  An  antennae  attached  to  one  of  the  backpack  harnesses  transmitted 


Figure  5.1 .4-1  Polestar  Technologies,  Inc.,  Magnetometer  System 
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to  the  PBS  for  navigation  purposes.  The  sensor  system  is  attached  to  a  computer  system  earned  by  a  third 
person;  this  individual  is  attached  to  the  magnetometer  by  a  flexible  safety  tube. 

According  to  Polestar’s  proposal,  the  PBS  uses  a  low  frequency  microwave  signal,  exhibiting  a  coverage 
range  of  3.0  kilometers  (1.9  miles).  Most  trees  do  not  interfere  with  its  long  wavelength  which  diffracts 
around  trees.  The  system’s  phase  detection  permits  spatial  resolution  to  a  fraction  of  a  wavelength,  thereby 
achieving  a  distance  measurement  accuracy  of  less  than  15  centimeters  (6  inches).  Established  monuments 
at  the  16A-hectare  area  were  used  as  base  stations,  and  triangulation  was  used  to  track  the  sensor  system. 
Although  Poles  tar  used  microwave  navigation  with  its  PBS,  the  system  was  also  equipped  to  use  a  GPS. 

Poles  tar  stated  that  the  navigation  data  are  stored  twice  per  second,  and  the  six  individual  axes  of  each 
magnetometer  are  read  at  50  Hz.  Proprietary  data  analysis  software  was  used  to  process  both  the  vector  and 
total  magnetic  field  data.  A  multiparameter,  nonlinear  least  squares  algorithm  was  used  to  accurately 
determine  x,  y,  and  z,  as  well  as  magnetic  mass,  inclination,  and  declination  (Polestar  1995a). 

5.1.4.2  System  Assessment 

This  section  summarizes  the  Polestar  demonstration,  based  on  observations  made  in  the  field. 

Requirements  for  Technology  Implementation 

Polestar  used  nine  people  to  complete  its  demonstration.  At  the  beginning  of  the  demonstration,  three 
beacons  that  made  up  the  PBS  were  placed  outside  of  the  1 6A-hectare  area  to  assist  in  navigation  on  the 
grid.  The  magnetometer  system  required  three  people  for  operation.  Polestar  used  two  teams,  alternating 
personnel  to  allow  for  rest  periods.  The  resting  team  and  the  remaining  three  people  were  responsible  for 
grid  layout. 

The  grid  was  laid  out  by  placing  stakes  every  3  meters  (10  feet)  along  the  grid  lines.  String  was  then  used  to 
visually  guide  the  survey  team  along  a  straight  path  from  these  stakes. 

All  system  equipment  used  for  this  demonstration  was  shipped  or  brought  to  JPG  by  Polestar.  Support 
equipment  used  by  Polestar  included  two  laptop  computers  to  store  and  process  the  data,  one  minivan  to 
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transport  personnel  and  supplies,  and  a  full-size  van  to  store  the  equipment  in  the  field.  Surveyor  stakes  and 
string  were  used  to  lay  out  lanes  within  the  grid  area.  The  support  trailer  supplied  electricity  to  recharge  the 
Polestar  system  batteries. 

Operational  Capabilities  and  Limitations 

Polestar’ s  system  appeared  to  operate  best  in  large  open  areas.  The  system  had  difficulty  accessing  wooded 
areas,  as  the  size  of  the  boom  precluded  maneuvering  around  more  than  one  or  two  trees  at  a  time.  In 
addition,  low  cover  in  woody  areas  presented  a  problem  because  the  PBS  antenna,  which  is  3.7  meters 
(12.0  feet)  high,  became  caught  in  the  overstory.  The  grid  layout  required  by  the  system  is  labor  intensive. 
The  survey  equipment  is  heavy  and  requires  alternating  operators  under  hot  and  humid  conditions. 

The  system  used  by  Polestar  required  little  maintenance  beyond  recharging  the  batteries;  delays  resulting 
from  battery  failure  resulted  in  a  total  of  44  minutes  of  downtime  during  the  demonstration.  Polestar 
experienced  no  other  delays  in  the  field  as  a  result  of  equipment  failure. 

5.1.43  Measured  Performance 

Polestar  surveyed  about  15  hectares  of  the  16A-hectare  area  with  its  magnetometers  in  32.5  of  the  allotted 
40  hours.  Polestar  reported  no  targets  within  the  16A-hectare  area.  In  its  final  report,  Polestar  stated  that 
the  solid-state  sensors  of  the  magnetometers  were  defective.  Polestar  stated  that  the  three  axes  of  the  sensor 
were  improperly  located  causing  a  “crosstalk.”  Polestar  noted  this  problem  early,  during  preprocessing  at 
JPG  but  thought  that  calibration  and  correction  could  be  used  to  restore  the  integrity  of  the  data.  However, 
the  defect  was  nonlinear  and  could  not  be  corrected.  The  only  data  provided  by  Polestar  were  the  navigation 
results,  illustrating  the  utility  of  the  PBS  for  field  use.  As  a  result,  no  data  analysis  was  possible  (Polestar 
1995b). 
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5.1.5 


Scintrex,  Inc. 


Sdntrex,  Inc.  (Scintrex),  demonstrated  from  July  12  through  17, 1995,  at  the  16A-hectare  area  at  JPG. 

5.1.5.1  Technology  Description 

Scintrex  used  two  magnetometers  called  Smartmags  for  its  demonstration  (see  Figure  5. 1.5-1).  The 
Smartmag  consists  of  a  staff-mounted  cesium-vapor  magnetometer  and  signal  processor,  a  memory  console, 
staff  mounted  display,  and  headphones;  and  a  belt-mounted  battery  pack  for  the  processor.  Three  different 
versions  of  the  Smartmag  exist:  die  sweep  version,  the  mapping  version,  and  the  survey  version,  all  using 
slightly  different  combinations  of  equipment.  The  mapping  version  was  used  for  this  demonstration,  and  it 
consisted  of  the  magnetometer  and  processor  with  a  belt-mounted  battery  pack  and  memory  console.  The 
magnetometer  sensor  consists  of  a  miniature  atomic  absorption  unit  from  which  a  signal  proportional  to  the 
intensity  of  the  ambient  magnetic  field  is  derived.  A  signal  processor  converts  the  signal  into  the  magnetic 
field  strength  in  nanoteslas  for  display  and  recording. 

A  Trimble  real-time  kinematic  GPS  was  coupled  with  the  Smartmag  to  position  the  survey  data.  A  Trimtalk 
reference  station  was  used  to  send  error  corrections  to  rover  units  in  the  field.  Total  field  magnetometer  data 
were  collected  every  0. 1  second,  with  GPS  position  updates  every  second.  Survey  lines  were  marked  with 
surveyor  staffs  placed  at  each  end  and  painted  at  1 -meter  intervals  to  ensure  that  areas  were  not  missed. 
Scintrex  personnel  walked  the  survey  lines  in  a  north-to-south  direction.  East-to-west  lines  were  used  to  tie 
the  surveyed  grid  cells  together. 

As  data  were  collected,  they  were  stored  in  the  system’s  memory  console  and  subsequently  transferred  to  a 
PC  for  permanent  storage.  A  reference  magnetometer  was  set  up  to  record  data  on  the  background  magnetic 
field  of  the  earth;  these  data  were  then  used  to  correct  the  magnetometer  data  collected  in  the  field.  The 
target  data  was  mapped  using  software  supplied  by  Geosoft,  Scintrex  subcontractors  (Scintrex  1995a). 

5.1 .5.2  System  Assessment 

This  section  summarizes  the  Scintrex  demonstration,  based  on  observations  made  in  the  field. 
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Figure  5.1 .5-1  Scintrex,  Inc.,  Smartmag  Magnetometer  System 


Requirements  for  Technology  Implementation 

Scintrex  used  four  people  to  complete  its  demonstration.  Two  people  operated  the  two  magnetometers  used 
for  the  survey.  One  person  supported  these  two  teams,  keeping  track  of  the  area  covered  and  transporting 
the  memory  console  to  the  support  trailer  for  downloading.  The  remaining  person,  from  Geosoft,  managed 
data  analysis,  which  took  place  at  the  support  trailer. 
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All  system  equipment  used  for  this  demonstration  was  shipped  or  brought  to  JPG  by  Scintrex.  Support 
equipment  used  by  Scintrex  included  several  laptop  computers  to  store  and  process  the  data,  one  minivan  to 
transport  personnel  and  supplies,  and  one  car  to  transport  personnel.  A  surveyor  staff  and  paint  lines  were 
used  for  navigation  through  the  grid  area.  Electricity  supply  was  provided  from  the  16A-hectare  support 
trailer  to  recharge  the  Scintrex  system  batteries. 

Operational  Capabilities  and  Limitations 

The  magnetometers  used  by  Scintrex  required  little  maintenance  beyond  recharging  the  batteries.  Because 
the  Smartmag  is  man-portable,  survey  efficiency  is  subject  to  the  physical  limitations  of  the  operator.  The 
data  acquisition  equipment  and  battery  pack  of  the  Smartmag  weigh  about  99  kilograms  (45  pounds), 
imposing  some  fitness  requirement  on  the  operator.  Although  the  man-portable  system  can  access  wooded 
areas  easily,  GPS  signals  were  often  lost  due  to  tree  cover.  As  a  result,  the  heavily  wooded  areas  of  the 
16A-hectare  area  were  not  surveyed. 

Several  delays  resulted  from  equipment  difficulties  during  the  demonstration.  The  GPS  took  longer  than 
expected  to  set  up  the  first  day,  and  the  cables  for  one  of  the  roving  GPS  units  operated  intermittently. 
Recurring  difficulties  in  maintaining  GPS  satellite  signals  or  “lock”  in  heavily  wooded  areas  also  delayed  the 
survey  considerably.  Generally,  four  to  eight  satellites  were  available.  When  the  operator  entered  a  heavily 
wooded  area,  the  cover  interfered  with  the  satellite  signal.  The  GPS  unit  would  not  function  unless  five 
satellites  were  available.  Once  the  satellite  lock  was  lost,  the  operator  stopped  the  magnetometer  and  waited 
until  the  lock  was  reestablished. 

One  of  the  two  magnetometer  data  recorders  had  extended  memory  that  lasted  for  8  hours.  However,  the 
other  data  recorder  had  only  standard  memory  configuration,  which  filled  up  in  2  hours  and  required 
downloading  several  times  daily.  Each  download  of  this  data  recorder  took  about  40  minutes,  and  as  a  result, 
the  system  was  not  functional  much  of  the  time.  The  two  Scintrex  personnel  operating  the  magnetometers 
had  difficulty  managing  the  operation,  navigation,  and  data  downloading  with  the  intermittent  support 
provided.  It  is  likely  that  more  of  the  area  could  have  been  covered  with  additional  personnel  and  equipment. 
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5.1.53 


Measured  Performance 


Scintrex  was  scored  on  5.76  hectares  (14.23  acres)  of  the  16A-hectare  area  (35.6  percent),  which  it  covered 
in  the  allotted  40  hours.  Although  Scintrex  covered  slightly  less  area  than  it  was  scored  on,  grid  cells  that 
were  covered  50  percent  or  more  were  included.  Scintrex  reported  255  targets  within  the  16A-hectare  area 
with  the  fence  line  area  removed  (see  Figure  5. 1.5-2)  (Scintrex  1995b). 

Scintrex’ s  performance  is  shown  in  Table  5. 1.5-1,  which  presents  the  following:  (1)  detection, 

(2)  localization,  and  (3)  classification  statistics  with  respect  to  type,  size,  and  class.  The  baseline  for 
computing  performance  included  both  ordnance  and  nonordnance  items.  The  detection  ratio  of  0.50  (cited  as 
PD  ordnance  in  Table  5. 1.5-1)  reflects  the  number  of  targets  detected  by  Scintrex  compared  to  the  total 
number  of  baseline  ordnance  targets  in  the  area  covered  within  the  16  hectares.  This  detection  probability  is 
significantly  different  than  the  probability  of  detection  arising  from  random  declarations  due  to  sensor  noise 
and  other  factors  ( P^mdom  =  0*06).  Scintrex  had  a  FAR  of  45.3  per  hectare. 

Scintrex’s  PD  of  0.50  fell  in  the  lower  half  of  the  results  for  magnetometer  systems.  These  results  may  be  in 
part  because  of  difficulties  in  the  field  in  maintaining  adequate  GPS  lock  as  well  as  the  inexperience  of  one 
of  the  two  equipment  operators.  Scintrex  also  had  the  second  highest  FAR  of  the  magnetometer 
demonstrators. 

For  most  demonstrators,  the  probability  of  ordnance  detection  depends  on  both  the  size  and  the  depth  of  the 
buried  ordnance  item.  Figure  5. 1.5-3  presents  a  scatter  plot  showing  detection  performance  as  a  function  of 
size  versus  depth  for  Scintrex.  Although  Scintrex  was  able  to  detect  several  large  targets,  small  and  medium 
targets  were  difficult  for  Scintrex  to  detect. 

The  localization  error  statistics  section  of  Table  5. 1.5-1  indicates  Scintrex’s  ability  to  estimate  the  location  of 
the  targets  declared.  Scintrex  reported  target  depths  between  0.13  and  4.76  meters  (0.43  and  15.62  feet) 
below  ground  surface.  Scintrex  provided  type,  size,  and  class  information  as  shown  in  Table  5.1 .5-1 . 
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Northing  (meters) 


Figure  5.1 .5-2 

Scintrex  Target  Declarations 


Easting  (meters) 
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TABLE  5.1.5-1 

PERFORMANCE  STATISTICS  FOR  SCINTREX 


Detection  Statistics 


Number  Baseline 

Number  Matched 

P»‘  ' . -  J 

Ordnance 

46 

23 

0.50 

Nonordnance 

23 

12 

0.52 

Total 

69 

35 

0.51 

p 

x  random 

0.06  1 

Number  False  Alarms 

232 

False  Alarm  Rate 

45.3/hectare 

False  Alarm  Ratio 

10.1 

Probability  False  Alarms 

0.0569 

Localization  Statistics 


1 -  - 

Mean  (m)b 

Std.  Dev.c(m)  (j 

||  Position  (x,y)  1 

dx 

-0.18 

0.74 

dy 

0.47 

0.65 

Radial 

0.94 

1  Depth  (z)  _ _ J 

d z? 

0.39 

0.79 

dz 

0.87 

Identification  and  Classification  Statistics 


Number  Baseline  Number  Detected  PDe 

Number  Correct 

mt 

II  Type  _ 

Ordnance 

58 

21 

0.36 

20 

0.95 

Nonordnance 

38 

16 

0.42 

0 

0.0 

Size  _ _ _ 

Large 

18 

8 

0.44 

4 

0.5 

Medium 

19 

9 

0.47 

9 

1.00 

Small 

21 

4 

0.19 

1 

0.25 

|  Class  II 

Bomb 

14 

8 

6 

0.75 

Projectile 

25 

10 

4 

0.40 

Mortar 

16 

3 

0.19 

1 

0.33 

Cluster 

3  1 

0 

0 

0 

NAs 

Notes:  *  Probability  of  detection  (based  on  Group  TM  A,  fence  area  excluded) 

b  Meter 

*  Standard  deviation 

d  Square  root  of  the  mean  square  depth  error 

*  Probability  of  detection  (based  on  closest  TMA,  fence  area  excluded) 
1  Probability  of  correctly  classifying  (based  on  closest  TMA) 

*  Not  applicable 
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Depth  (meters) 


Figure  5.1 .5-3 
Scintrex  Detection  Ability 


Small  Medium  Large 


Principal  Diameter  (millimeters) 


■  Target  Detected 
□  Target  Not  Detected 
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5.1.6 


Vallon  GmbH 


Vallon  GmbH  (Vallon),  in  cooperation  with  Security  Search  Products  and  Sales,  demonstrated  from  July  12 
through  17,  1995,  at  the  16B-hectare  area  at  JPG.  Vallon  also  participated  in  Phase  I  of  the  UXO  ATD 
program. 


5.1.6.1  Technology  Description 

Vallon  used  three  magnetometer  systems  during  the  demonstration:  a  vehicle-towed  system  and  two 
separate  man-portable  systems.  The  vehicle-towed  system  used  by  Vallon  was  the  MSV-5  multisensor 
vehicle 

(see  Figure  5. 1 .6-1 ).  This  system  uses  an  ATV  to  tow  a  sensor  platform  with  an  array  of  five  magnetometers 
(EL1302A1  ferrous  locators)  spaced  0.5  meter  (1.6  feet)  apart.  Vallon  describes  theEL1302Al  as  a  high 
sensitivity  differential  magnetometer  used  to  detect  iron.  Each  sensor  is  connected  to  a  separate 


Figure  5.1.6-1  Vallon  GmbH  Multisensor  Vehicle 
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microcomputer  (MCI)  for  data  acquisition.  The  MSV-5  is  ridden  by  an  operator,  who  keeps  the  unit  on 
course  while  the  unit  is  towed  by  the  ATV.  A  differential  GPS  provides  navigation  and  GPS  tagging  for 
detected  targets  with  accuracy  of  less  than  1  meter  (3  feet).  To  provide  accurate  direction  and  distance 
control,  the  MSV-5  operator  uses  a  radio  link  to  the  GPS  navigation  system.  The  MSV-5  is  intended  to 
provide  accurate  positional  information  with  continuous  recording. 

The  second  system  used  by  Vallon  is  called  the  man-portable  detection  system  (MANPODS).  This  system 
consists  of  a  single  sensor  magnetometer,  an  operator  backpack  containing  a  data  acquisition  microcomputer 
(MCI),  and  a  differential  GPS  for  accurate  position  information.  Vallon  used  two  MANPODS  at  JPG;  each 
were  used  in  inaccessible  areas  where  the  MSV-5  could  not  be  used. 

The  third  system  used  by  Vallon  was  its  man-portable  sensor  positioning  system  (SEPOS)  (see  Figure 
5.1 .6-2).  This  system  is  identical  to  the  MANPODS,  except  that  GPS  was  not  employed.  In  instances  when 
MANPODS  could  not  receive  satellite  signals,  Vallon  marked  the  starting  positions  with  GPS  and  then  used 
survey  tapes  to  conduct  the  survey.  The  SEPOS  survey  tapes  were  placed  manually.  The  SEPOS  lines  were 


Figure  5.1.6-2  Vallon  Man-Portable  Sensor  Positioning  System 
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100  meters  (328  feet)  long  and  were  marked  at  each  meter  with  a  marker  detector.  Each  marker  was 
recorded  by  the  MCI .  Vallon  used  the  grid  system  for  navigation  with  the  SEPOS.  Vallon  walked  north  to 
south  lines  within  die  grid  cells  using  survey  tapes  to  mark  its  location  and  coverage. 

The  GPS  included  a  base  receiver,  which  was  positioned  on  a  known  benchmark  in  the  field,  one  differential 
receiver  on  the  MSV-5,  one  receiver  on  each  MANPODS,  and  two  radio  repeaters  for  the  differential  radio 
link.  As  described  above,  the  SEPOS  did  not  use  GPS  except  to  mark  the  starting  position  for  the  survey. 
Vallon  developed  an  algorithm  to  receive  data  from  the  MCI  units  and  to  analyze  and  process  this  data  to 
produce  target  lists  and  field  maps  (Vallon  1995a). 

Vallon  indicated  the  following  improvements  from  Phase  I:  (1)  use  of  differential  GPS  for  navigation  and 
GPS  tagging  of  detected  targets,  (2)  a  radio  link  between  the  driver  of  the  ATV  and  the  GPS  navigation  for 
direction  and  distance  control,  (3)  balancing  of  the  MSV-5  for  control  over  ragged  terrain,  and  (4)  increased 
storage  capabilities  for  the  MCI  (Vallon  1995a). 

5.1.6.2  System  Assessment 

This  section  summarizes  the  demonstration  by  Vallon,  based  on  observations  made  in  the  field. 

Requirements  for  Technology  Implementation 

Vallon  used  nine  people  to  complete  its  demonstration.  Two  people  operated  the  MSV-5;  one  person  drove 
the  ATV,  while  the  other  person  steered  and  operated  the  system  sensors  from  behind.  Six  people  operated 
either  the  MANPODS  or  the  SEPOS,  alternating  personnel  to  provide  rest  periods.  The  remaining  person 
supervised  the  operation. 

Vallon’s  sensors  and  GPS  equipment  were  shipped  to  JPG.  Vallon  purchased  an  ATV  locally  for  use  with 

the  MSV-5.  Vallon  rented  a  large  truck  to  transport  the  ATV,  MSV-5,  and  MANPODS  equipment.  Two 
minivans  were  rented  to  transport  personnel  and  equipment  to  and  from  the  site.  Additional  equipment 
required  included  a  laptop  PC  for  data  analysis,  four  radios  for  communication  in  the  field,  and  survey  stakes 
and  tapes  for  marking  location  in  the  field.  Replacement  bolts,  wooden  support  brackets,  batteries,  and  a  flat 
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tire  inflator  and  sealer  were  purchased  locally  for  repairs  to  the  MSV-5  and  ATV.  Vallon  required  an 
electrical  supply  to  recharge  batteries;  this  was  provided  at  the  support  trailer. 

Operational  Capabilities  and  Limitations 

The  MSV-5  and  GPS  can  cover  about  0.6  to  0.8  hectare  (1 .5  to  2  acres)  per  hour  depending  on  the  terrain. 
The  data  recording  for  this  system  is  continuous,  which  prevented  data  downloading  delays.  However,  poor 
satellite  conditions  caused  some  delays,  and  the  MSV-5  experienced  numerous  delays  due  to  equipment 
failure.  The  ATV  that  towed  the  MSV-5  had  a  flat  tire,  as  well  as  recurring  engine  problems.  Two  bolts 
also  sheared  off  the  MSV-5  during  the  demonstration  and  had  to  be  replaced.  The  MSV-5  is  designed  for 
areas  that  are  generally  clear  of  high  foliage  and  large  obstacles  (such  as  trees).  As  a  result,  the  MSV-5  had 
difficulty  operating  in  areas  of  uneven  terrain  and  maneuvering  the  turnarounds  (particularly  at  the 
beginning  of  the  survey). 

Both  the  MANPODS  and  the  SEPOS  operated  independently  of  the  MSV-5.  The  MANPODS  were  not  used 
to  the  degree  anticipated  because  Vallon  was  unable  to  maintain  GPS  signals  due  to  the  tree  cover.  Instead, 
Vallon  used  the  SEPOS  (which  was  used  during  Phase  I)  in  an  effort  to  increase  its  coverage.  Both  the 
MANPODS  and  then  the  SEPOS  were  used  simultaneously  with  the  MSV-5. 

All  three  of  the  magnetometer  systems  used  by  Vallon  employed  the  same  sensor  and  data  analysis 
techniques.  The  major  differences  between  the  systems  were  with  the  sensor  configuration  and  the 
navigation  and  target  locating  subsystems. 

5. 1.6.3  Measured  Performance 

Vallon  was  scored  on  8.83  hectares  (21.81  acres)  of  the  16B-hectare  area  (52.6  percent)  for  the  combined 
systems  during  the  allotted  40  hours.  Although  Vallon  covered  slightly  more  area  than  it  was  scored  on, 
only  grid  cells  covered  50  percent  or  more  were  included  in  the  scoring.  Vallon  provided  results  for  both  the 
GPS  and  the  SEPOS,  the  data  from  these  two  systems  were  combined  for  this  data  analysis.  Vallon  reported 
a  total  of  1,903  targets  with  the  GPS  and  SEPOS  within  the  8.83  hectares  it  was  scored  on,  once  the  fence 
line  area  was  removed  (see  Figure  5. 1.6-3)  (Vallon  1995b). 
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Vallon’s  performance  is  shown  in  Table  5. 1.6-1,  which  presents  the  following:  (1)  detection,  (2) 

localization,  and  (3)  classification  statistics  with  respect  to  type,  size,  and  class.  The  baseline  for  computing 

detection  performance  included  both  ordnance  and  nonordnance  items.  The  detection  ratio  of  0.57  (cited  as 

PD  ordnance  in  Table  5.1.6-1)  reflects  the  number  of  targets  detected  by  Vallon  as  compared  to  the  total 

number  of  baseline  ordnance  targets  in  the  area  covered.  The  probability  of  detection  arising  from  random 

declarations  due  to  sensor  noise  and  other  factors  was  0.25  (P^om)  for  Vallon,  the  highest  of  all  T 

demonstrators.  Vallon  had  a  FAR  of  225.9  per  hectare,  also  the  highest  of  all  the  demonstrators. 

• 

As  part  of  Phase  I,  Vallon  had  a  PD  of  0.72  for  ordnance  items.  The  FAR  was  1 49  per  hectare.  A 
comparison  of  Phase  I  and  Phase  II  detection  performance  shows  that  Vallon  obtained  a  lower  PD  value  for 
Phase  II  detection  performance  data,  0.57  compared  to  0.72  for  Phase  I.  Correspondingly,  Vallon’s  FAR 
also  increased,  from  149  to  225.9. 

For  most  demonstrators,  the  probability  of  ordnance  detection  depends  on  both  the  size  and  depth  of  the 
buried  ordnance  item.  Figure  5. 1 .6-4,  which  is  a  scatter  plot  showing  detection  performance  as  a  function  of 
size  versus  depth,  illustrates  this  relationship  for  Vallon.  Vallon  was  somewhat  successful  at  detecting 
targets  of  all  sizes  at  various  depths. 

The  localization  error  statistics  section  in  Table  5. 1.6-1  indicates  Vallon’s  ability  to  estimate  the  locations  of 
the  targets  declared.  Vallon  reported  target  depths  between  0  and  6.27  meters  (0  and  20.57  feet)  below 
ground  surface.  Vallon  did  not  provide  ordnance  type  information,  declaring  all  target  detections  as 
“unknown.”  Vallon  did  provide  size  and  class  information. 
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TABLE  5.1.6-1 

PERFORMANCE  STATISTICS  FOR  VALLON 


Detection  Statistics 


Number  Baseline 

Number  Matched 

Pd*  1 

Ordnance 

47 

27 

0.57 

Nonordnance 

5 

1 

0.20  1 

Total 

52 

28 

0.54 

P 

x  random 

0.25  1 

Number  False  Alarms 

r_  1,849  1 

False  Alarm  Rate 

225.9/hectare 

False  Alarm  Ratio 

68.5 

Probability  False  Alarms 

0.2838 

Localization  Statistics 


i  _ 

Mean  (m)b 

Std.  Dev.c(m)  1 

1  Position  (x,y)  _  - - 1 

dx 

-0.01 

0.59 

dy 

0.13 

0.78 

Radial 

0.83 

Depth  (z)  _ _ _ 1 

dz? 

0.02 

0.99 

dz 

0.98 

Identification  and  Classification  Statistics 


i 

Number  Baseline 

Number  Detected 

ESI 

Number  Correct 

mat 

Type 

Ordnance 

65 

31 

0.48 

0 

0 

Nonordnance 

7 

1 

0.14 

0 

0 

II  Size 

■■ 

Large 

11 

6 

0.55 

1 

Medium 

14 

8 

0.57 

7 

0.88 

Small 

38 

16 

0.42 

6 

0.38 

II  Class 

Bomb 

5 

3 

0.60 

1 

0.33 

Projectile 

31 

16 

0.52 

0 

0 

Mortar 

25 

9 

0.36 

0 

0 

Cluster 

4 

3 

0.75 

0 

0 

Notes:  *  Probability  of  detection  (based  on  Group  TMA,  fence  area  excluded) 

b  Meter 

c  Standard  deviation 

"  Square  root  of  the  mean  square  depth  error 

*  Probability  of  detection  (based  on  closest  TMA,  fence  area  excluded) 
r  Probability  of  correctly  classifying  (based  on  closest  TMA) 
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Depth  (meters) 


Figure  5.1. 6-4 
Vallon  Detection  Ability 


Small  Medium  L 


■  Target  Detected 
□  Target  Not  Detected 


5.2 


ELECTROMAGNETIC  INDUCTION  SENSOR  SYSTEMS 


5.2.1  Bristol  Aerospace  Ltd. 

Bristol  Aerospace  Ltd.  (Bristol),  demonstrated  from  June  14  through  18, 1995,  at  the  16A-hectare  area  at 
JPG. 


5.2.1. 1  Technology  Description 

Bristol  used  a  remotely  operated  ATV  that  towed  a  pulse  induction  sensor  array  (see  Figure  5.2.1-1). 

Bristol  describes  the  sensor  platform  as  able  to  detect  electromagnetic  signatures.  The  sensor  uses  pulse 
induction  coils  to  detect  the  presence  of  conductive  metallic  objects.  Data  are  transmitted  to  a  remote-control 
command  station  for  recording  and  analysis.  At  the  command  station,  the  sensor  data  is  plotted  on  a  map  of 
the  area  being  scanned  using  color  intensity  to  show  signal  strength.  A  differential  GPS  receiver  mounted 


Figure  5.2.1 -1  Bristol  Aerospace  Ltd.,  Pulse  Induction  Sensor  Array 
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on  the  vehicle  provides  a  continuous  readout  of  the  location  of  the  vehicle.  Bristol  stated  in  its  proposal  that 
the  vehicle  location  provided  by  the  GPS  is  accurate  to  plus  or  minus  1  meter  (3.3  feet). 

When  a  detection  is  made  by  the  sensor  platform,  the  system  automatically  marks  the  position  provided  by 
GPS  and  alerts  the  operator  in  the  command  station.  The  electromagnetic  signature  of  the  object  is  displayed 
on  the  command  station  monitor,  and  the  location  in  universal  transverse  mercator  (UTM)  coordinates  and 
detection  rime  are  automatically  recorded.  A  video  camera  is  mounted  on  the  vehicle  with  a  pan  and  tilt 
mechanism;  the  video  signal  is  sent  back  to  the  operator’s  monitor  in  the  command  station.  The  vehicle  is 
controlled  autonomously  via  GPS  in  open  terrain,  but  requires  operator  guidance  around  obstacles;  guidance 
is  provided  via  joystick  controls  and  a  television  monitor. 

The  towed  sensor  platform  contains  two  sensor  coils;  each  coil  is  2-meters  (6.6-feet)  wide.  A  towing 
“carpet”  attaches  the  platform  to  the  ATV  and  carries  the  necessary  cables  and  hoses  to  the  system 
controller.  The  detector  system  automatically  unhitches  from  the  ATV  if  the  towed  sensor  platform  becomes 
obstructed. 

The  Bristol  system  functions  by  electromagnetic  pulses  from  the  transmitter  coils  that  generate  eddy  currents 
in  the  buried  metal;  these  currents  in  turn  produce  a  magnetic  field  around  the  object.  When  the  decay  of  the 
magnetic  field  is  detected,  it  induces  a  voltage  in  the  receiver  coils.  The  received  voltage  results  in  a  signal 
that  is  sent  to  the  operator  console  for  processing.  The  detector  coils  create  a  zone  of  sensitivity  such  that 
any  metallic  object  that  passes  within  its  influence  is  detected  (Bristol  1995a). 

5.2.1.2  System  Assessment 

This  section  summarizes  the  demonstration  by  Bristol,  based  on  observations  made  in  the  field. 
Requirements  for  Technology  Implementation 

Bristol  used  five  people  to  complete  its  demonstration.  One  to  two  people  observed  the  progress  of  the 
sensor  platform.  The  remaining  personnel,  located  in  the  command  station,  remotely  operated  the  system 
and  monitored  and  recorded  its  progress. 
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Bristol  shipped  the  majority  of  its  equipment  to  JPG  from  Canada.  A  flatbed  trailer  was  used  to  transport 
the  Bristol  system  including  all  the  spare  parts  and  supplies  required  for  operation  and  maintenance.  Tins 
equipment  included  a  command  station  that  contained  all  computer  equipment  and  monitors;  a  smaller  trailer 
that  contained  two  gasoline-powered  generators  serving  as  electrical  supply;  and  the  sensor  platform, 
including  the  ATV  used  for  towing  the  system.  Locally  rented  equipment  included  an  additional  ATV  used 
to  transport  personnel  around  the  perimeter  of  the  site  as  well  as  a  crane,  which  was  used  to  move  the  control 
station  on  and  off  the  site.  Additional  equipment,  acquired  locally,  included  miscellaneous  hardware  needed 
to  replace  the  broken  hitch  connecting  the  array  to  the  vehicle. 

Operational  Capabilities  and  Limitations 

Bristol’s  ATV  can  move  at  a  rate  of  about  10  kilometers  per  hour  (6  miles  per  hour).  The  system  covered 
the  open  areas  autonomously,  while  the  operator  remotely  guided  the  ATV  around  obstacles.  The  system 
performs  best  in  open  terrain,  due  both  to  the  nature  of  the  remote-controlled  system  for  guidance  around 
obstacles  and  the  size  of  the  sensor  platform  The  wide  turning  radius  of  the  vehicle  limits  maneuverability 
around  trees  and  in  making  turns  at  the  end  of  a  swath  line.  The  2-  to  4-meter-  (6.6-  to  1 3-foot-)  wide  sensor 
platform  is  unable  to  access  densely  wooded  areas  because  it  cannot  fit  between  closely  spaced  trees.  In 
addition,  uneven  terrain  and  ground  cover  caused  the  detector  system  to  unhitch  from  the  ATV .  The  system 
experienced  minor  delays  until  the  detector  system  was  reattached  to  the  vehicle;  however,  the  detachable 
configuration  prevented  damage  to  the  sensor  platform  from  brush  or  trees. 

Numerous  delays  occurred  as  a  result  of  equipment  failures.  Mechanical  problems  included  sensor  failure,  a 
broken  bolt  on  the  hitch,  cable  connection  failures,  a  bumed-out  electrical  switch,  hydraulic  system  in-line 
filter  replacement,  and  steering  control  failure.  Despite  about  1 1  hours  of  downtime  resulting  from 
mechanical  problems,  Bristol  surveyed  most  of  the  acreage  thoroughly.  To  increase  coverage  around  trees, 
the  survey  was  performed  in  both  north-to-south  and  east-to-west  directions. 

5.2.1.3  Measured  Performance 

Bristol  was  scored  on  13.58  hectares  (33.56  acres)  of  the  16A-hectare  area  (83.9  percent)  covered  with  its 
ATV-towed  system  during  the  allotted  40  hours.  Although  Bristol  covered  slightly  more  area  than  it  was 
scored  on,  only  grid  cells  covered  50  percent  or  more  were  included.  Figure  5.2.1-2  shows  Bristol’s  target 


5-44 


declare™™  at  the  16A-hectare  area  (Bristol  1995b).  Bristol  reported  a  total  of  566  targets  with  the  sensor 
platform  within  the  13.58  hectares  (33.56  acres),  with  the  fence  line  area  removed.  Bristol’s  performance  is 
shown  in  Table  5.2.1-1,  which  presents  detection  and  localization  statistics.  Bristol  did  not  provide  type, 
size,  or  class  information  so  classification  statistics  were  not  computed. 

The  baseline  for  computing  detection  performance  included  both  ordnance  and  nonordnance  items.  The 
detection  ratio  of  0.62  (PD  ordnance)  reflects  the  number  of  targets  detected  by  Bristol  as  compared  to  the 
total  number  of  baseline  ordnance  targets  in  the  area  covered  (Bristol  1 995b).  This  detection  probability  is 
significantly  different  than  the  probability  of  detection  arising  from  random  declarations  due  to  sensor  noise 
and  other  factors  (Pnmdom  =  0.05).  Bristol  had  a  FAR  of  38.2  per  hectare. 

For  most  demonstrators,  the  probability  of  ordnance  detection  depends  on  both  the  size  and  depth  of  the 
buried  item.  Figure  5.2. 1-3  presents  a  scatter  plot  showing  detection  performance  as  a  function  of  size 
versus  depth  that  illustrates  this  relationship  for  Bristol.  Bristol  was  successful  at  detecting  some  targets  of 
all  sizes  at  a  variety  of  depths. 

The  localization  error  statistics  section  of  Table  5.2.1-1  indicates  Bristol’s  ability  to  estimate  the  location  of 
the  targets  declared.  Bristol  reported  all  target  depths  at  0. 10  meter  (0.33  feet)  below  ground  surface. 
Bristol  did  not  provide  type,  size,  or  class  information. 
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Northing  (meters) 


Figure  5.2.1-2 

Bristol  Target  Declarations 
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TABLE  5.2.1-1 

PERFORMANCE  STATISTICS  FOR  BRISTOL 


Detection  Statistics 


1 - 

Number  Baseline 

Number  Matched 

Pd* 

|  Ordnance 

114 

71 

0.62 

|  Nonordnance 

33 

19 

0.58  S 

1  Total 

147 

90 

0.61  1 

0.05  1 

Number  False  Alarms 

495  ”1 

False  Alarm  Rate 

38.2/hectare 

False  Alarm  Ratio 

6.97 

Probability  False  Alarms 

0.048 

Localization  Statistics 


i - - - 

Mean  (m)b 

Std.  Dev.c(m)  || 

H  Position  (x,y)  _ _ _ J 

dx 

-0.30 

0.80 

dy 

0.10 

0.80 

Radial 

1.04 

1  Depth  (z)  _ _ _ _ J 

d 3d 

-0.66 

0.72 

1  Idz 

0.97 

Identification  and  Classification  Statistics 


|  Number  Baseline 

Number  Detected  1 

PDe 

Number  Correct 

El 

Type 

Ordnance 

141 

64 

0.45 

0 

0 

Nonordnance 

55 

29 

0.53 

0 

0 

Size 

Large 

30 

13 

0.43 

0 

Medium 

50 

25 

0.50 

MM 

Small 

60 

26 

0.43 

0 

0 

H  Class 

Bomb 

20 

8 

0.40 

0 

0 

Projectile 

61 

31 

0.51 

0 

0 

Mortar 

52 

21 

0 

!  Cluster 

8 

4 

0.50 

0 

Notes:  *  Probability  of  detection  (based  on  Group  TMA,  fence  area  excluded) 

b  Meter 

c  Standard  deviation 

d  Square  root  of  the  mean  square  depth  error 

e  Probability  of  detection  (based  on  closest  TMA,  fence  area  excluded) 
f  Probability  of  correctly  classifying  (based  on  closest  TMA) 
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Figure  5.2.1 -3 
Bristol  Detection  Ability 


Small  Medium  Large 


Principal  Diameter  (millimeters) 


■  Target  Detected 
□  Target  Not  Detected 
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5.2.2 


GeoPotential 


GeoPotential  demonstrated  from  August  23  through  27,  1995,  at  the  16A-hectare  area  at  JPG. 

5.2.2. 1  Technology  Description 

GeoPotential  used  three  electromagnetic  induction  systems  (Aqua-Tronics  A6  tracers)  during  the 
demonstration  (see  Figure  5.2.2-1).  One  electromagnetic  system  was  equipped  with  a  datalogger 
(OMNIDATA  PRO  2000).  Each  electromagnetic  system  operates  at  1 17  kilohertz  (kHz)  and  contains  both 
a  transmitting  antenna  and  a  receiving  antenna,  which  are  separated  by  a  1 .22 -meter-  (4.00-foot-)  long 
handle.  The  system  operates  by  transmitting  an  electromagnetic  wave  into  the  ground.  Conductive  objects 
in  the  subsurface  cause  changes  in  the  wave,  which  then  show  up  as  a  voltage  variation  in  the  receiver.  The 
power  output  varies  from  17  to  270  volts. 

The  electromagnetic  system  can  be  used  in  search  mode,  mapping  mode,  or  a  combination  of  the  two. 

Search  mode  uses  audio  and  visual  outputs  to  map  the  location  of  subsurface  objects  in  real  time.  The 


Figure  5.2.2- 1  GeoPotential  Electromagnetic  Induction  Instrument 
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operator  then  marks  the  location  with  paint  or  stakes,  and  the  coordinates  are  determined  later.  The 
electromagnetic  system  mapping  mode  uses  the  A6  tracer  and  datalogger  to  acquire  and  record 
electromagnetic  measurements  used  to  generate  profiles  and  contour  maps.  Data  are  acquired  along  profiles 
at  1 .5-second  intervals  when  recorded  by  the  datalogger.  Profile  data  are  then  downloaded  from  the 
datalogger  to  a  portable  computer  where  they  are  edited,  gridded,  and  contoured  to  produce  an 
electromagnetic  profile  and  contour  maps.  Data  are  interpreted  to  determine  the  nature  of  a  buried  object. 
The  combined  search  and  mapping  modes  consist  of  using  the  search  mode  to  initially  locate  objects  then  lay 
out  profiles  over  the  object.  The  mapping  mode  is  then  used  to  record  electromagnetic  data  from  the  object 
for  quantitative  analysis  (GeoPotential  1995a). 

52.2.2  System  Assessment 

This  section  summarizes  the  demonstration  by  GeoPotential,  based  on  observations  made  in  the  field. 

Requirements  for  Technology  Implementation 

GeoPotential  used  six  people  to  complete  its  demonstration.  The  three  electromagnetic  systems  were 
operated  by  three  one-person  teams.  A  two-person  team  measured  the  locations  of  the  anomalies  or  items 
located  by  each  electromagnetic  system.  The  remaining  individual  analyzed  the  data  and  produced  the 
contour  maps.  All  system  equipment  used  for  this  demonstration  was  driven  to  JPG  by  GeoPotential 
personnel.  Support  equipment  required  to  perform  the  geophysical  survey  included  a  laptop  computer  and 
printer  used  to  analyze  data  and  produce  contour  maps.  GeoPotential  had  additional  batteries  for  its  survey 
equipment  and  did  not  require  on-site  recharging  of  batteries.  Data  analysis  was  performed  on  a 
GeoPotential  work  station  temporarily  set  up  in  the  on-site  support  trailer.  Navigation  equipment  used  by 
GeoPotential  included  an  optical  transit,  a  stadia  rod,  a  measuring  wheel,  and  flagging  material. 

Operational  Capabilities  and  Limitations 

The  electromagnetic  system  required  very  little  setup  or  takedown  time.  The  system  was  operational  within 
15  minutes,  and  about  the  same  amount  of  time  was  needed  to  store  equipment  at  the  end  of  each  day. 
GeoPotential  reported  that  its  system  is  limited  by  weather,  such  as  electrical  storms,  which  would  disrupt 
readings,  and  extremely  cold  weather,  which  would  drain  batteries. 


The  GeoPotential  proposal  stated  that  it  intended  to  use  the  electromagnetic  system  in  mapping  mode  to 
generate  electromagnetic  contour  maps.  However,  when  GeoPotential  began  processing  data,  the  field  data 
indict?*!  that  ambient  electromagnetic  noise  caused  drifting  and  level  shifts  in  the  electromagnetic  data.  The 
noise  degraded  the  resulting  electromagnetic  contour  maps,  and  GeoPotential  decided  that  the  combination 
mode  would  be  more  effective.  As  a  result,  two  of  the  three  systems  were  used  in  combination  mode  to 
survey  the  area. 

Navigation  across  the  grid  was  completed  using  the  grid  coordinate  system  on  the  16A-hectare  area  and  was 
aided  by  spray  paint  dashes  made  by  Geophex  Ltd.,  (which  demonstrated  during  the  same  time  period)  at 
1 ,5-meter  (5-foot)  intervals  in  an  east-west  direction.  Surveying  was  completed  in  east-west  fashion.  Line- 
of-sight  traverses  were  made  between  flags  during  data  acquisition.  All  objects  located  were  surveyed  to  the 
nearest  grid  stake  with  a  compass  on  a  stadia  rod  and  a  tape  line.  Without  the  preexisting  grid  system  at  the 
16A-hectare  area,  GeoPotential  would  have  needed  to  set  up  its  own  grid  system  to  track  its  coverage  and 
measure  the  location  of  the  detected  anomalies. 

5.2.2  J  Measured  Performance 

GeoPotential  covered  13.60  hectares  (33.61  acres)  of  the  16A-hectare  area  (84.0  percent)  with  the 
electromagnetic  systems  in  about  38.5  hours  (of  40  hours  allotted).  Two  electromagnetic  systems  covered 
about  74  percent  of  this  covered  area  using  the  combination  mode.  The  electromagnetic  system  with  the 
datalogger  covered  the  remaining  26  percent  of  the  covered  area  in  the  mapping  mode;  however,  the  resulting 
contour  maps  were  deemed  unacceptable  for  data  interpretation  and  were  not  submitted  for  scoring. 
GeoPotential  resurveyed  about  1.3  hectares  (3.2  acres)  of  the  area  as  part  of  its  quality  assurance  program. 
Areas  previous  covered  with  the  datalogger  system  were  resurveyed,  and  the  datalogger  system  was  used  to 
profile  the  anomalies  detected  by  the  other  two  systems.  In  this  way,  GeoPotential  was  able  to  compare  data 
and  contour  maps  (GeoPotential  1995b). 

GeoPotential  reported  168  targets  within  the  13.60  hectares  that  it  was  scored  on  (see  Figure  5.2.2-2).  Only 
grid  cells  that  were  covered  50  percent  or  more  were  scored.  GeoPotential’ s  performance  with  the 
electromagnetic  system  is  shown  in  Table  5.2.2-1,  which  presents  the  following:  (1)  detection, 

(2)  localization,  and  (3)  classification  statistics  with  respect  to  type,  size,  and  class. 
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The  baseline  for  computing  detection  performance  included  both  ordnance  and  nonordnance  items.  The 
detection  ratio  of  0. 1 1  (cited  as  PD  ordnance  in  Table  5.2.2-1)  reflects  die  number  of  targets  detected  by 
GeoPotential  as  compared  to  the  total  number  of  baseline  ordnance  targets  in  the  area  they  covered. 
GeoPotential ’s  probability  of  detection  based  on  only  random  declarations  (Pnmdom)  is  0.02.  GeoPotential  had 
a  FAR  of  12.0  per  hectare.  Of  the  three  EM  sensors  demonstrated,  GeoPotential  had  the  lowest  PD  and  FAR. 

For  most  demonstrators,  the  probability  of  ordnance  detection  depends  on  both  the  size  and  depth  of  the 
buried  ordnance  item.  Figure  52.2-3  presents  a  scatter  plot  showing  detection  performance  as  a  function  of 
size  versus  depth  that  illustrates  this  relationship  for  GeoPotential.  This  figure  shows  no  clear  demarcation 
for  detection  as  a  function  of  size  and  depth.  All  targets  were  difficult  for  GeoPotential  to  detect.  The 
localization  error  statistics  section  of  Table  5.2.2-1  indicates  GeoPotential’ s  ability  to  estimate  the  location 
of  the  targets  declared.  GeoPotential  reported  target  depths  between  0.01  and  3.50  meters  (0.03  and  1 1 .48 
feet)  below  ground  surface.  Table  5.2.2-1  also  indicates  GeoPotential’s  type,  size,  and  class  capabilities,  all 
of  which  were  provided. 
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TABLE  5.2.2-1 

PERFORMANCE  STATISTICS  FOR  GEOPOTENTIAL 


Detection  Statistics 


Number  Baseline 

Number  Matched 

Pd* 

Ordnance 

113 

12 

0.11 

Nonordnance 

29 

3 

0.10 

Total 

142 

15 

0.11 

P 

A  rtnrfnm 

0.02 

Number  False  Alarms 

156 

False  Alarm  Rate 

12.0/hectare 

False  Alarm  Ratio 

13.0 

Probability  False  Alarms 

0.0151 

Localization  Statistics 


|  Mean  (m)b  j 

Std.  Dev.c(m)  1 

V  Position  (x,y)  _ _ _ — - N 

dx 

0.44 

0.99 

dy 

-0.37  ' 

0.86 

Radial 

1.30  ~| 

Depth  (z)  _ _ _ — - 

dz* 

-0.38 

0.73 

dz 

0.80 

II 

iUiimuiva  nwu 

Number  Baseline 

Number  Detected 

Pd* 

m 

Type  _ — - - - i 

Ordnance 

142 

12 

12 

1.00 

Nonordnance 

50 

3 

0.06 

0 

0 

L_ - 

Size 

_ _ _ __ _ _ _ . _ _ _ _ _ “1 

Large 

29 

5 

0 

Medium 

49 

4 

0.08 

2  1 

0.50 

Small 

63 

3 

1 

0.33 

Class 

r  Bomb 

16 

4 

0.25 

2 

0.50 

Projectile 

63 

5 

0.08 

3 

0.60 

Mortar 

54 

3 

0.06 

1 

0.33 

Cluster 

9 

0 

m 

0 

NAB 

Notes:  *  Probability  of  detection  (based  on  Group  TMA,  fence  area  excluded) 

b  Meter 

c  Standard  deviation 

d  Square  root  of  the  mean  square  depth  error 

e  Probability  of  detection  (based  on  closest  TMA,  fence  area  excluded) 
'  Probability  of  correctly  classifying  (based  on  closest  TMA) 

8  Not  applicable 
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Figure  5.2. 2-3 

GeoPotential  Detection  Ability 


Small  Medium  Large 


Principal  Diameter  (millimeters) 
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5.2.3  Parsons  Engineering  Science,  Inc. 

Parsons  Engineering  Science,  Inc.  (Parsons),  demonstrated  from  May  31  through  June  4,  1995,  at  the  16A- 
hectare  area  at  JPG. 

5.2.3.1  Technology  Description 

Parsons  used  an  electromagnetic  induction  system  with  both  field  and  laboratory  components.  The  field 
components  included  Geonics  EM-6 1  high  sensitivity  electromagnetic  time-domain  metal  detectors  (see 
Figure  5. 2.3-1),  associated  dataloggers,  and  a  laptop  PC  for  review  and  storage  of  the  survey  data.  The 
computer  laboratory  component  for  data  analysis  included  Intergraph’s  MGE  Grid  Analyst. 

Parsons  used  three  EM-6 1  units  to  conduct  a  geophysical  survey  of  the  site.  The  EM-6 1  unit  is  an 
electromagnetic  induction  profiler.  This  type  of  instrument  measures  and  records  the  conductivity  of  nearby 


Figure  5.23-1  Parsons  Engineering  Science,  Inc.,  EM-61  Electromagnetic  Unit 
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materials.  A  sinusoidal  waveform  is  emitted  from  the  transmitter,  causing  eddy  currents  in  the  subsurface. 
The  intensity  of  the  eddy  currents  is  a  function  of  the  ground  conductivity  and  the  conductivity  of  buried 
material.  The  eddy  currents  cause  a  time-varying  secondary  electromagnetic  field  that  is  measured  at  the 
receiver.  The  EM-61  unit  differs  from  the  frequency  domain  electromagnetic  instrument  in  that  no  voltage 
component  is  included.  Therefore,  the  entire  signal  measured  results  from  the  electromagnetic  eddy  current 
induced  by  the  primary  field.  Parsons  stated  in  its  proposal  that  the  EM-61  generates  electromagnetic  pulses 
at  150  times  per  second  and  records  the  measured  electromagnetic  field  between  each  pulse.  This 
configuration  allows  enough  time  for  the  response  in  the  conductive  earth  to  dissipate  before  measuring  the 
prolonged  response  of  the  buried  metal.  Measurements  recorded  by  the  datalogger  are  transferred  to  the 
laptop  PC  for  processing. 

After  Parsons  collected  the  data  at  the  site,  this  information  was  provided  to  their  subcontractor,  Sanford, 
Cohen,  and  Associates  (SC  &  A)  for  data  interpretation.  SC  &  A  performed  data  analysis  using  the 

t 

Huntsville  UXO  Knowledgebase  (UXO-KB)  developed  for  the  U.S.  Army  Corps  of  Engineers,  Huntsville 
Division.  The  UXO-KB  allows  for  variations  of  signal  characteristics  within  a  range  of  acceptable  values 
and  allows  the  range  of  acceptable  values  to  change  daily  as  more  data  are  added  to  the  system 
(Parsons  1995a). 

5.23.2  System  Assessment 

This  section  summarizes  the  Parsons  demonstration,  based  on  observations  made  in  the  field. 

Requirements  for  Technology  Implementation 

Parsons  used  eight  people  to  complete  its  demonstration.  The  three  EM-61  units  were  operated  by 
two-person  teams.  On  each  team,  one  individual  operated  the  datalogger  and  instrument,  while  the  second 
individual  was  responsible  for  record  keeping  and  navigation.  One  person  was  responsible  for  data  analysis, 
which  took  place  at  the  support  trailer.  The  remaining  individual  served  as  the  field  supervisor. 

All  system  equipment  used  for  this  demonstration  was  shipped  or  brought  to  JPG  by  Parsons.  Support 
equipment  used  by  Parsons  included  two  laptop  computers  to  store  and  process  the  data,  two  minivans  to 
transport  personnel  and  supplies,  and  a  panel  truck  to  store  the  assembled  equipment  in  the  field.  Surveyor 
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tapes,  pin  flags,  and  string  were  used  for  navigation  through  the  grid  area.  Pin  flags  were  used  to  mark  the 
beginning  and  end  of  a  swath  line,  while  the  string  was  used  to  guide  the  teams  along  a  straight  path.  The 
support  trailer  supplied  electricity  to  recharge  the  Parsons  system  batteries. 

Operational  Capabilities  and  Limitations 

The  EM-6 1  units  used  by  Parsons  required  little  maintenance  beyond  recharging  the  batteries.  Parsons 
experienced  no  delays  due  to  equipment  failure.  According  to  Parsons,  its  system  is  limited  to  a  survey  speed 
of  0.6  meter  per  second  (2.0  feet  per  second).  Heavy  brush  and  heavily  wooded  areas  can  further  slow  the 
survey  speed  by  decreasing  maneuverability.  Extremely  rugged  terrain  may  require  the  EM-6 1  antenna 
system  to  be  carried  using  a  shoulder  harness,  but  this  was  not  necessary  at  JPG.  Such  uneven  or  wooded 
terrain  adversely  affects  the  system  by  hampering  the  accurate  location  of  comers  and  flag  locations 
designating  lanes.  Parsons  also  indicated  that  the  system  is  unable  to  operate  in  standing  water  greater  than 
15  centimeters  (6  inches)  deep  without  modification.  Although  some  areas  of  standing  water  were  present 
firing  the  JPG  demonstration,  Parsons  did  not  indicate  that  it  made  any  modifications  to  accommodate  these 
wet  areas.  Parsons  stated  that  surface  interferences  such  as  wire  fences  can  affect  data  quality  within  a  3-  to 
4-meter  (10-  to  13-foot)  radius  of  the  system  (Parsons  1995b).  Navigation  across  the  grid  was  completed 
using  the  grid  coordinate  system  on  the  1 6A-hectare  area. 

5.2.4.3  Measured  Performance 

Parsons  surveyed  the  entire  16A-hectare  area  with  its  EM-61  units  in  the  allotted  40  hours.  Parsons  reported 
602  targets  within  die  16A-hectare  area,  once  the  fence  line  area  was  removed  (see  Figure  5.2.3-2)  (Parsons 
1995b).  Parsons’  performance  is  shown  in  Table  5 .2.3-1,  which  presents  the  following:  (1)  detection, 

(2)  localization,  and  (3)  classification  statistics  with  respect  to  type,  size,  and  class. 

The  baseline  for  computing  detection  performance  included  both  ordnance  and  nonordnance  items.  The 
detection  ratio  of  0.85  (cited  as  PD  ordnance  in  Table  5.2.3-1)  reflects  the  number  of  targets  detected  by 
Parsons  compared  to  the  total  number  of  baseline  ordnance  targets  in  the  16  hectares.  This  detection 
probability  is  significantly  different  than  the  probability  of  detection  arising  from  random  declarations  due  to 
sensor  noise  and  other  factors  (P^om  =  0.05).  Parsons  had  a  FAR  of  32.5  per  hectare. 
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Parsons  had  the  highest  PD  of  all  the  systems  demonstrated  for  Phase  II.  Parsons  teamed  with  SC  &  A  to 
perform  the  data  processing  which  may,  in  part,  account  for  their  performance  relative  to  the  other  systems 
demonstrated. 

For  most  demonstrators,  the  probability  of  ordnance  detection  depends  on  both  the  size  and  depth  of  the 
buried  ordnance  item.  Figure  5. 2.3-3,  which  is  a  scatter  plot  showing  detection  performance  as  a  function  of 
size  versus  depth,  illustrates  this  relationship  for  Parsons.  Parsons  was  able  to  detect  targets  of  all  sizes  and 
generally  appeared  to  detect  all  but  the  deepest  targets  in  each  category. 

The  localization  error  statistics  section  of  Table  5. 2.3-1  indicates  Parsons’  ability  to  estimate  the  location  of 
the  targets  declared.  Parsons  reported  target  depths  ranged  between  0  and  3  meters  (0  and  10  feet)  below 
ground  surface.  Parsons  provided  type,  size,  and  class  information  as  shown  in  Table  5.2.3-1. 
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Northing  (meters) 


Figure  523-2 
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TABLE  523-1 

PERFORMANCE  STATISTICS  FOR  PARSONS 


Detection  Statistics 


1 

Number  Baseline 

Number  Matched 

PDa 

Ordnance 

127 

108 

0.85 

Nonordnance 

41 

22 

0.54 

Total 

168 

130 

0.77 

P 

0.05  1 

Number  False  Alarms 

505 

False  Alarm  Rate 

32.5/hectare 

False  Alarm  Ratio 

4.68 

Probability  False  Alarms 

0.0408 

Localization  Statistics 


1 - = - 

Mean  (m)b 

Std.  Dev.c(m)  | 

|  Position  (x,y)  1 

dx 

-0.03 

0.73 

dy 

0.14 

0.54 

Radial 

0.79 

9  Depth  (z)  _ _ 

dz4 

-0.24 

0.68 

Idz 

0.72 

Identification  and  Classification  Statistics 


i 

Number  Baseline 

Number  Detected 

PDe 

m 

Ordnance 

158 

109 

0.69 

85 

0.78 

Nonordnance 

65 

30 

3 

0.10 

Large 

35 

25 

0.71 

4 

0.16 

Medium 

53 

40 

0.75 

30 

0.75 

Small 

69 

44 

0.64 

10 

0.23 

U  Class  H 

Bomb 

21 

17 

0.81 

2 

Projectile 

69 

47 

0.68 

27 

Mortar 

59  1 

40 

0.68 

10 

_025| 

Cluster 

9 

5 

Notes:  *  Probability  of  detection  (based  on  Group  TM  A,  fence  area  excluded) 

b  Meter 

‘  Standard  deviation 

"  Square  root  of  the  mean  square  depth  error 

'  Probability  of  detection  (based  on  closest  TM  A,  fence  area  excluded) 
f  Probability  of  correctly  classifying  (based  on  closest  TMA) 
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Figure  5.23-3 
Parson  Detection  Ability 


Small  Medium  Large 


■  Target  Detected 
□  Target  Not  Detected 
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5.3 


GROUND  PENETRATING  RADAR  SENSOR  SYSTEMS 


53.1  Airborne  Environmental  Surveys,  Inc. 

Airborne  Environmental  Surveys,  Inc.  (AES),  demonstrated  from  May  3 1  through  June  3,  1995,  at  the 
32-hectare  area  at  JPG.  AES  also  participated  in  Phase  I  of  the  UXO  ATD  program. 

5.3.1.1  Technology  Description 

AES  used  an  EMS-20  airborne  ground  penetrating  radar  (AGPR)  system  (see  Figure  5. 3. 1-1).  The  AGPR 
system  is  mounted  on  a  Bell  412  helicopter.  The  503-MHz  center-frequency  AGPR  system  is  coupled  to 
two  oppositely  polarized,  bistatic,  helical  antennae.  The  system  control,  data  collection,  and  GPS  navigation 
hardware  are  mounted  in  the  passenger  compartment.  With  the  oppositely  polarized  antenna  configuration, 
ground  penetration  in  dry  sod  should  be  at  least  5  meters,  and  responses  from  surface  clutter  should  be 
highly  attenuated.  The  EMS-20  radar  transmissions  are  analog,  these  analog  signals  received  by  the 


Figure  53.1-1  Airborne  Environmental  Surveys,  Inc.,  Airborne  Ground  Penetrating 
Radar  System 
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antennae  are  digitized  in  an  on-board  computer  and  stored  on  the  video  channel  of  a  VHS  tape  via  a  pulse- 
code  modulator  recorder.  The  helicopter  flight  path  is  navigated  by  a  Trimble  3000F  Navigator,  while  a  six- 
channel  Trimble  Pathfinder  Professional  with  MC-V  (rover)  is  used  to  record  GPS  positioning  data  of  the 
helicopter  flight  path.  AES  subcontracted  Coleman  Research  Corporation  to  analyze  the  GPR  data. 

At  JPG,  AES  combined  data  from  its  AGPR  with  a  63-channel  digital  airborne  imaging  spectrometer 
(DAIS).  AES  subcontracted  to  Geophysical  and  Environmental  Research  Corporation  (GER)  for  the  DAIS 
surveys.  The  GER  DAIS  was  flown  aboard  a  Piper  Navajo  airplane  (see  Figure  5.3. 1-2).  The  DAIS  is  an 
airborne  unit  designed  to  gather  spectral  information  for  environmental  studies,  geologic  mapping,  research, 
and  other  applications.  A  Kennedy  type  scanner  is  used  to  acquire  the  images,  which  are  formed  at  the 
entrance  slit  to  the  spectrometer.  The  spectrometer  has  four  wavelengths  ranging  from  0.35  to  12.5  microns. 
The  locations  of  anomalies  interpreted  from  these  data  were  determined  by  referencing  the  images  with 
ground  markers.  Before  DAIS  data  analysis,  data  are  subject  to  preprocessing  or  correcting,  which  consist  of 


gyro  and  baseline  corrections,  inversion  of  thermal  channels,  panoramic  correction,  and  time-constant 
correction  (AES  1995a). 

AGPR  system  improvements  from  Phase  I  include  a  12-channel  base  station  and  on-board  GPS  systems  for 
differential  correction  of  recorded  GPS  data  and  for  recording  positioning  data  of  the  GPR,  respectively,  and 
a  new  time  recording  system  taken  directly  from  the  GPS  to  eliminate  the  possibility  of  drift  (AES  1995a). 

53.1.2  System  Assessment 

This  section  summarizes  the  demonstration  by  AES,  based  on  observations  made  in  the  field  and  information 
provided  by  AES. 

Requirements  for  Technology  Implementation 

AES  used  five  people  to  complete  its  demonstration  at  JPG.  Two  additional  people  from  GER  were  on  site 
to  conduct  the  DAIS  surveys.  Operation  of  the  AGPR  system  required  two  AES  personnel  on  site  to  direct 
the  helicopter  with  orange  flags,  while  three  other  AES  personnel  were  in  the  helicopter.  The  DAIS  surveys 
were  conducted  at  sunrise,  midday,  and  after  sunset  to  profile  thermal  activity  throughout  the  day.  AES 
personnel  were  often  not  on  site  during  the  DAIS  surveys. 

Support  equipment  required  to  perform  the  survey  included  a  Bell  412  helicopter  and  a  Piper  Navajo 
airplane,  equipped  with  the  system-required  instrumentation.  Additional  equipment  needed  for  the  survey 
included  a  optical  color  video;  a  12-channel  Trimble  Navigation  GPS  Pathfinder  Community  Base  Station 
for  differential  correction  of  recorded  GPS  data;  and  a  laptop  computer.  White  bed  sheets  were  purchased  by 
GER  personnel  and  placed  in  four  locations  throughout  the  area.  The  bed  sheets  were  used  as  reference 
points  for  the  images  produced  from  the  DAIS.  AES  later  surveyed  the  locations  of  the  bed  sheets  using 
GPS  coordinates;  this  approach  enabled  data  from  the  two  systems  to  be  compared  and  evaluated. 

Operational  Capabilities  and  Limitations 

AES  completed  the  AGPR  survey  of  the  32-hectare  area  in  about  4  hours.  The  entire  area  was  surveyed 
from  an  altitude  of  38  meters  (125  feet)  above  the  ground  surface,  in  a  cross-grid  pattern  with  flight  line 
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centers  separated  by  8  meters  (26  feet).  GER’s  DAIS  system  covered  relatively  large  areas  within  a  short 
time.  GER  covered  the  32-hectare  area  in  0.5  hour,  but  additional  or  redundant  data  sets  were  collected  to 
ensure  full  coverage  and  data  quality.  Inclement  weather,  such  as  heavy  fog  and  thunderstorms,  caused 
several  delays  during  the  demonstration. 

The  AES  system  is  limited  by  terrain  clutter  and  soil  moisture.  Precipitation  occurred  on  several  days  during 
the  demonstration,  and  standing  water  was  often  present  over  much  of  the  ground  surface.  Rainfall  during 
the  first  3  weeks  of  May  was  reportedly  300  percent  above  normal.  The  AES  system  does  not  differentiate 
between  buried  ordnance  and  other  detected  targets  such  as  nonordnance  or  empty  holes,  and  standing  water 
and  wet  conditions  greatly  decrease  the  effectiveness  of  the  radar.  The  sensors  used  in  this  demonstration 
were  not  useful  in  determining  the  weight,  type,  or  class  of  detected  objects.  Most  of  the  objects  were 
classified  as  large  because  of  the  lower  limitation  of  resolution. 

53.13  Measured  Performance 

AES  covered  the  entire  32-hectare  area  with  the  two  systems  in  16  of  the  allotted  24  hours.  AES  identified  a 
total  of  37  target  locations  at  the  32-hectare  area.  Figure  5.3.1-3  shows  AES’s  target  declarations  at  the 
32-hectare  area,  with  the  fence  line  area  removed.  Table  5.3. 1-1  shows  AES’s  results  which  includes  the 
following:  (l)detection,  (2)  localization,  and  (3)  classification  statistics  with  respect  to  type,  size,  and  class. 

The  baseline  for  computing  detection  performance  included  both  ordnance  and  nonordnance  items.  The 
detection  ratio  (PD  ordnance)  of  0.05  reflects  the  number  of  targets  detected  by  AES  as  compared  to  the  total 
number  of  baseline  ordnance  targets  in  the  area  covered  with  the  fence  line  area  removed  (AES  1995b).  This 
detection  probability  is  not  significantly  different  than  the  probability  of  detection  arising  from  random 
declarations  due  to  sensor  noise  and  other  factors  (P^don =  0.01).  AES  had  a  FAR  of  0.9  per  hectare. 

AES’s  results  were  comparable  to  those  of  other  airborne  demonstrators  as  they  showed  little  of  no 
capability  detecting  ordnance.  In  addition,  AES’s  results  are  comparable  to  the  results  from  other  GPR 
systems  from  Phase  I  and  Phase  II. 

As  part  of  Phase  I,  AES  had  a  PD  of  0.0 1  for  ordnance  items.  The  FAR  was  1 .29  per  hectare.  A  comparison 
of  Phase  I  and  Phase  II  detection  performance  shows  that  AES  obtained  slightly  higher  PD  values  for  Phase  II 
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(0.05)  detection  performance  data.  Similarly,  AES’s  FAR  improved,  decreasing  from  1 .29  in  Phase  I  to  0.9 
in  Phase  n,  although  these  differences  are  not  statistically  significant. 

For  most  demonstrators,  the  probability  of  ordnance  detection  depends  on  both  the  size  and  depth  of  the 
buried  ordnance  item.  Figure  5.3. 1-4,  which  is  a  scatter  plot  showing  detection  performance  as  a  function  of 
size  versus  depth,  illustrates  this  relationship  for  AES.  The  figure  shows  no  relationship  between  these 
parameters  for  AES,  which  appeared  to  have  difficulty  in  assessing  depth  at  all. 

The  localization  error  statistics  section  of  the  table  indicates  AES’s  ability  to  estimate  the  locations  of  the 
targets  declared.  AES  reported  target  depths  between  0.01  and  1.80  meters  (0.03  and  5.91  feet)  below 
ground  surface.  AES  provided  ordnance  type,  size,  and  class  information,  as  shown  in  the  table. 


5-67 


TABLE  53.1-1 

PERFORMANCE  STATISTICS  FOR  AES 


Detection  Statistics 


1 - - — 

Number  Baseline 

Number  Matched 

Pd* 

Ordnance 

201 

9 

0.05 

Nonordnance 

20 

0  1 

Total 

221 

9 

0.04 

P 

A  random 

0.01  1 

Number  False  Alarms 

28 

False  Alarm  Rate 

0.9/hectare 

False  Alarm  Ratio 

3.11 

Probability  False  Alarms 

0.0069 

Localization  Statistics 


|  Mean  (m)b  j 

Std.  Dev.c(m)  f 

II  Position  (x,y)  H 

dx 

-0.42 

2.68 

dy 

-0.26 

2.09 

Radial 

2.76 

II  Depth  (z)  . 

dz? 

-0.49 

0.91 

dz 

0.99 

Identification  and  Classification  Statistics 


1 

Number  Baseline 

Number  Detected 

PDe 

Number  Correct 

■211 

I  Type  _ _ _ _ _ II 

|  Ordnance 

245 

9 

0.04 

8 

0.89 

1  Nonordnance 

27 

0 

0 

0 

NA* 

U  Size 

Large 

41 

4 

3 

0.75 

Medium 

37 

2 

0 

Small 

165 

3 

0.02 

1 

0.33 

1  Class  II 

Bomb 

23 

1 

0.04 

1 

1.00 

Projectile 

73 

6 

0.08 

6 

1.00 

Mortar 

143 

2 

0.01 

Cluster 

6 

0 

0 

0 

NA  | 

Notes:  *  Probability  of  detection  (based  on  Group  TMA,  fence  area  excluded) 

b  Meter 

c  Standard  deviation 

d  Square  root  of  the  mean  square  depth  error 

e  Probability  of  detection  (based  on  closest  TMA,  fence  area  excluded) 
f  Probability  of  correctly  classifying  (based  on  closest  TMA) 

8  Not  applicable 
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Depth  (meters) 


Figure  5J.1-4 
AES  Detection  Ability 


Small  Medium  Large 


Principal  Diameter  (millimeters) 


■  Target  Detected 
□  Target  Not  Detected 
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5.3.2 


Kaman  Sciences  Corporation 


Kaman  Sciences  Corporation  (Kaman)  demonstrated  from  August  9  through  14, 1995,  at  the  16A-hectare 
area  at  JPG. 


5.3.2.1  Technology  Description 

Kaman  used  a  towed  ground  penetrating  radar  (GPR)  mounted  on  a  fiberglass  and  plexiglass  sled  (see 
Figure  5.3 .2-1).  The  system  is  mounted  on  a  four-wheel-drive  truck,  with  the  antennas  towed  on  a  wheeled 
sled.  Kaman  describes  its  GPR  system  as  a  nonintrusive,  range-gated  synthetic  pulse  radar  system.  The 
system  uses  separate  transmitting  and  receiving  antennas  capable  of  detecting  metallic  objects  and 
identifying  characteristic  changes  below  ground  surface.  The  Kaman  GPR  system  uses  a  frequency-stepped, 
wide-band  radar  capable  of  detecting  metallic  objects.  Kaman  stated  in  its  proposal  that  it  expected  target 
resolution  on  the  order  of  0.1  meter  (3.94  inches)  for  depth  (Kaman  1995a). 


Figure  5 .3 .2-1  Kaman  Sciences  Corporation  Towed  Ground  Penetrating  Radar 
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Frequencies  were  generated  by  a  Hewlett-Packard  network  analyzer  combined  with  a  digital  quadrature 
detector  (DQD)  giving  the  GPR  receiver  a  range  of  between  1  and  3,000  MHz.  The  optimal  range  for  the 
system  operating  at  JPG  was  between  100  and  800  MHz.  The  DQD  combined  with  a  range  gate  removes 
direct  feedover  between  the  transmitter  and  receiver  as  well  as  strong  unwanted  reflections.  The  GPR 
operates  at  a  dynamic  range  of  greater  than  200  decibels.  A  differential  GPS  was  used  for  navigation. 
Differential  GPS  data  was  used  for  location  scanning,  grid  layout,  and  vehicle  guidance.  The  remote 
differential  GPS  was  placed  and  calibrated  for  use  at  monument  1  on  the  grid. 

Kaman  used  the  following  subcontractors  for  its  technology  demonstration:  John  Hopkins  University 
Applied  Physics  Laboratory,  Program  for  Milj oovervlkingsteknikk,  and  SUSAR,  Inc.  Kaman  used 
discrimination  algorithms  developed  by  SUSAR  to  post-process  and  analyze  the  collected  data.  Kaman  and 
SUSAR  use  a  proprietary  technique  named  the  target  adaptive  matched  illumination  radar  (TAMIR)  system, 
which  is  intended  to  eliminate  false  radar  signals.  Kaman  stated  in  its  proposal  that  TAMIR  has  the  ability 
to  differentiate  between  UXO  and  nonordnance  items. 

53.2.2  System  Assessment 


This  section  summarizes  the  demonstration  by  Kaman,  based  on  observations  made  in  the  field.. 


Requirements  for  Technology  Implementation 

Kaman  used  six  people  to  complete  its  demonstration.  One  person  drove  the  ATV,  while  two  people 
accompanied  it,  helping  to  guide  the  system  on  the  grid.  Two  people  were  responsible  for  data  downloading 
and  analysis.  The  sixth  person  set  up  and  maintained  the  GPS  system. 

Kaman  drove  its  equipment  to  JPG  for  the  demonstration.  Kaman  intended  to  survey  the  area  with  its 
four-wheel-drive  pickup  truck,  however,  the  truck  became  stuck  in  the  wet  clay  soils  on  the  morning  of  the 
first  day  and  had  to  be  removed  with  a  tow  truck.  Kaman  then  rented  a  John  Deere  6-by-4-wheel  ATV 
locally  to  survey  the  site.  Two  four-wheel-drive  pickup  trucks  were  used  to  transport  personnel  and  supplies 
and  to  store  the  assembled  equipment  in  the  field. 
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Additional  support  equipment  required  to  perform  the  geophysical  survey  included  two  laptop  computers  to 
store  and  process  data,  one  gas-powered  generator  to  operate  the  GPR/GPS  system,  and  one  1 .2-meter- 
(4.0-foot-)  wide  sled  that  held  the  GPR  transmitter  and  receiver.  Surveyor  stakes  and  tapes,  pin  flags,  and 
tape  measures  were  used  for  guidance  through  the  grid,  which  was  surveyed  in  an  east-west  direction.  The 
support  trailer  supplied  electricity  to  recharge  the  GPR  batteries. 

Operational  Capabilities  and  Limitations 

As  described  above,  the  system  proposed  for  use  by  Kaman  proved  unusable  due  to  the  wet  conditions  at 
JPG.  Kaman’s  GPR  system,  using  the  ATV  as  a  tow  vehicle,  has  a  rate  of  up  to  about  1  meter  per  second 
(3  feet  per  second).  The  system  performs  best  in  open  terrain.  Heavy  brush  and  low  areas  in  the  terrain 
slowed  the  survey  considerably  due  to  access  difficulties.  Kaman  did  not  survey  in  the  heavily  wooded  areas 
of  the  grid  because  of  the  difficult  access  as  well  as  the  loss  of  GPS  reception  in  these  areas.  Kaman  was 
also  unable  to  survey  in  wet  low  areas  because  the  GPR  pulse  is  rapidly  degraded  by  standing  water  or  high 
soil  moisture  content.  While  in  the  field,  Kaman  personnel  remarked  that  the  clay  soil  might  present  a 
problem  for  GPR  detection  of  metal  objects. 

Several  short  delays  resulted  from  equipment  failures.  Kaman  experienced  a  loose  electrical  connector  with 
one  battery  terminal,  which  caused  some  data  loss.  The  ATV  became  stuck  on  a  stump,  which  also  caused  a 
delay.  In  addition,  about  every  2  hours,  the  GPR/GPS  system  had  to  be  shut  down  for  about  5  minutes  so 
that  the  field  generator  could  be  refueled. 

5 3.2 3  Measured  Performance 

Kaman  was  scored  on  8.5 1  hectares  (2 1 .03  acres)  of  the  1 6A-hectare  area  (52.6  percent)  covered  with  its 
vehicle  towed  system  during  the  allotted  40  hours.  Although  Kaman  covered  slightly  more  area  than  it  was 
scored  on,  only  grid  cells  covered  50  percent  or  more  were  included.  Figure  5.3.2-2  shows  Kaman’s  target 
declarations  at  the  16A-hectare  area  (Kaman  1995b).  Kaman  reported  a  total  of  32  targets  with  its  system 
within  the  8.51  hectares  (21.03  acres)  scored.  Kaman’s  performance  is  shown  in  Table  5.3.2-1,  which 
presents  both  detection  and  localization  statistics.  Kaman  did  not  provide  type,  size,  and  class  information, 
so  classification  statistics  were  not  computed. 
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The  baseline  for  computing  detection  performance  included  both  ordnance  and  nonordnance  items.  The 
detection  ratio  of  0  (cited  as  PD  ordnance  in  Table  5.3.2-1)  reflects  the  number  of  targets  detected  by  Kaman 
as  compared  to  the  total  number  of  baseline  ordnance  targets  in  the  area  covered.  Kaman  did  not  detect,  and 
therefore  were  unbable  to  classify,  any  of  the  65  ordnance  items  or  14  nonordnance  items  that  were  in  the 
area  they  surveyed.  Kaman  had  a  FAR  of  4.2  per  hectare.  Kaman’s  results  are  comparable  to  the  results 
from  other  GPR  systems  from  Phase  I  and  Phase  II. 

For  most  demonstrators,  the  probability  of  ordnance  detection  depends  on  both  the  size  and  depth  of  the 
buried  ordnance  item.  Figure  5.3.2-3,  which  is  a  scatter  plot  showing  detection  performance  as  a  function  of 
size  versus  depth,  illustrates  this  relationship  for  Kaman.  This  figure  shows  that  Kaman  had  difficulty 
detecting  targets  of  all  sizes  across  the  spectrum  of  depths. 

Due  to  the  lack  of  correct  declarations,  the  mean  radial  accuracy  for  the  targets  Kaman  declared  could  not  be 
determined.  Kaman  reported  target  depths  at  0.50  to  2.50  meters  (1 .64  to  8.20  feet)  below  ground  surface. 
Again,  due  to  the  results,  the  mean  depth  error  could  not  be  determined. 
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Northing  (meters) 


Figure  5 32-2 

Kaman  Target  Declarations 
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TABLE  5.3 2-1 

PERFORMANCE  STATISTICS  FOR  KAMAN 


Detection  Statistics 


Number  Baseline 

Number  Matched 

p„-  -  n 

Ordnance 

65 

0 

0 

Nonordnance 

14 

0 

0 

Total 

79 

0 

_ <> _ zj 

U  ^ranrinm 

0.01 

Number  False  Alarms 

33 

False  Alarm  Rate 

4.2/hectare 

False  Alarm  Ratio 

NA8 

Probability  False  Alarms 

0.0053 

Localization  Statistics 


i - - 

Mean  (m)b 

Std.  Dev.c(m)  1 

1  Position  (x,v)  - 1 

dx 

0 

0 

dy 

0 

0 

Radial 

0 

Depth  (z)  _ _ _ _ 

d 2? 

0 

0 

Id z 

0 

Identification  and  Classification  Statistics 


1 - 

Number  Baseline 

Number  Detected 

wm 

Number  Correct 

mm 

Type 

Ordnance 

77 

0 

0 

0 

NA 

Nonordnance 

23 

0 

0 

0 

NA 

Size 

Large 

14 

0 

0 

0 

Medium 

31 

0 

0 

0 

NA 

Small 

31 

0 

0 

0 

NA 

Hi 

Bomb 

9 

0 

0 

0 

Projectile 

31 

0 

0 

0 

NA 

Mortar 

32 

0 

0 

0 

NA 

Cluster 

5 

0 

0 

0 

NA 

Notes:  *  Probability  of  detection  (based  on  Group  TMA,  fence  area  excluded) 

b  Meter 

c  Standard  deviation 

d  Square  root  of  the  mean  square  depth  error 

*  Probability  of  detection  (based  on  closest  TMA,  fence  area  excluded) 
f  Probability  of  correctly  classifying  (based  on  closest  TMA) 

*  Not  applicable 
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Depth  (meters) 


Figure  53.2-3 
Kaman  Detection  Ability 


Small  Medium  Large 


■  Target  Detected 
□  Target  Not  Detected 
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533 


SRI  International 


SRI  International  (SRI)  demonstrated  from  July  26  through  30, 1995,  at  the  32-hectare  area  at  JPG.  SRI 
also  participated  in  Phase  I  of  the  UXO  ATD  program.. 

5 3 3.1  Technology  Description 

SRI  demonstrated  an  airborne  ground  penetrating  radar  (GPR)  system.  The  GPR  system  is  mounted  on  a 
Jetstream  3 1  aircraft  ( Figure  5.3.3-1).  The  transmitters  and  antennas  are  housed  in  a  modified  luggage  pod 
on  the  underside  of  the  aircraft.  The  system  control,  data  collection,  and  GPS  navigation  hardware  are 
mounted  in  two  racks  in  the  passenger  compartment.  SRI  incorporated  an  ultra-wide  bandwidth  to 
compensate  for  the  decreased  resolution  associated  with  the  long  wavelengths.  SRI  also  used  a  low 
frequency  radar  (150-500  MHz)  in  conjunction  with  the  GPR  to  increase  soil  penetration.  The  system  used 
a  combination  of  polarizations,  both  horizontal  and  vertical,  to  reduce  the  effects  of  surface  clutter.  The 
system  uses  three-dimensional  processing  and  on-board  navigation  using  GPS  information.  Positional  data 
are  obtained  with  a  differential  GPS. 


The  SRI  system  integrates  radar  returns  from  a  linear  range  of  aircraft  positions.  The  system  generates  a 
two-dimensional  image  that  shows  regions  of  high  and  low  radar  return.  The  physics  of  the  radar-soil 
interactions  cause  the  buried  targets  to  appear  farther  away  from  the  aircraft  than  they  really  are. 


Figure  533-1  SRI  International  Airborne  Ground 
Penetrating  Radar  System 
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The  amount  of  the  offset  depends  on  the  depth  of  bunal  and  the  characteristics  of  the  soil.  During  Phase  I, 
SRI  used  a  multi-angle  processing  approach  to  determine  depth,  with  poor  results.  For  Phase  II,  SRI  used  a 
stereoscopic  viewing  approach  by  analyzing  synthetic  aperture  radar  (SAR)  data.  This  approach  generates 
three-dimensional  images.  This  three-dimensional  processing  is  intended  to  reduce  the  terrain  limitations 
inherent  in  the  radar  system  (SRI  1 995a). 

SRI  system  improvements  from  Phase  I  included  the  use  of  dual  polarization  radar  horizontal-horizontal 
(HH)  and  vertical-vertical  (W),  as  apposed  to  only  HH  polarization  for  Phase  I.  A  second  major  advance 
was  the  development  of  three-dimensional  processing. 

533.2  System  Assessment 

This  section  summarizes  the  demonstration  by  SRI,  based  on  observations  made  in  the  field  and  information 
provided  by  SRI. 

Requirements  for  Technology  Implementation 

SRI  used  six  people  to  complete  its  demonstration.  Two  people  were  on  site  during  setup  and  takedown;  one 
of  these  persons  was  on  site  during  flight  times.  The  remaining  four  people  were  either  on  board  the  aircraft 
during  the  survey  or  at  the  airport.  Support  equipment  required  to  perform  the  survey  included  a  Jetstream 
aircraft  equipped  with  the  system-required  instrumentation.  Additional  equipment  needed  for  the  survey 
included  a  laptop  computer  with  a  color  monitor;  a  GPS  receiver;  a  radio  unit  for  communication  between  air 
and  ground  personnel;  a  camera  and  tripod  for  photographing  panoramic  landscape  features;  and  a  hand-held 
GPS  45. 

SRI  used  four  large  aluminum  comer  reflectors  to  enhance  the  system’s  ability  to  identify  ground  features. 
These  reflectors  were  assembled  on  site  during  the  first  day  of  the  demonstration;  assembly  took  8  hours  to 
complete.  Tools  used  in  assembling  the  reflectors  were  purchased  locally.  A  local  person  was  hired  to  assist 
the  SRI  ground  crew  during  setup  and  takedown  periods.  A  truck  was  required  to  transport  the  reflectors  on 
site  once  they  were  assembled.  A  utility  truck  was  rented  locally  and  used  to  transport  the  reflectors  and 
other  equipment  during  the  demonstration. 
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Operational  Capabilities  and  Limitations 


SRI’s  system  is  able  to  cover  relatively  large  areas  within  a  short  time.  SRI  covered  the  32-hectare  area  in 
2  hours,  but  additional  or  redundant  data  sets  were  collected  to  ensure  full  coverage  and  data  quality.  The 
site  was  surveyed  at  altitudes  ranging  from  564  to  1 ,027  meters  (1,850  to  3,370  feet)  above  mean  sea  level. 
JPG  is  located  about  80.8  meters  (265  feet)  above  sea  level. 

SRI  indicated  that  its  system  is  limited  by  terrain  clutter,  targets  buried  greater  than  1 .0  meter  (3.3  feet) 
below  ground  surface,  and  soil  moisture.  The  SRI  system  does  not  differentiate  between  ordnance  and  other 
detected  targets.  Standing  water  and  wet  conditions  greatly  decrease  the  effectiveness  of  the  radar.  In 
addition ,  the  SRI  system  cannot  determine  ordnance  type  or  size. 

System  limitations  with  regard  to  elevation-induced  anomalies  can  be  attributed  to  the  nature  of  synthetic 
aperture  imaging  with  a  side-looking  radar.  According  to  SRI,  SAR  images  are  intrinsically  distorted 
because  the  angular  sweep  of  the  radar  reflections  is  mapped  onto  a  flat  plane.  SRI’s  image  generation 
process  accounts  for  this,  but  it  assumes  that  the  target  area  is  perfectly  level  and  smooth,  which  in  practice 
is  never  the  case,  according  to  SRI.  As  a  result,  the  SAR  image  is  affected  by  additional  distortion  and  by 
layover,  which  means  that  with  severe  topography  the  top  of  the  terrain  is  closer  to  the  aircraft  than  the 
bottom  of  the  terrain.  Use  of  the  comer  reflectors  in  the  survey  area  lessens  this  distortion.  Although  the 
topography  at  JPG  is  not  severe,  the  comer  reflectors  were  used  to  enhance  ground  features  (SRI  1995b). 

53.3 3  Measured  Performance 

SRI  covered  the  entire  32-hectare  area  within  1 8  of  the  allotted  24  hours.  SRI  did  not  use  the  setup  day 
(Tuesday),  choosing  instead  to  set  up  on  the  first  day  of  the  demonstration  period  (W ednesday).  SRI 
identified  a  total  of  86  potential  UXO  locations  at  the  32-hectare  area,  with  the  fence  line  area  removed  (see 
Figure  5.3.3-2)  (SRI  1995b). 

SRI  provided  the  surface,  or  apparent,  locations  in  the  database  along  with  the  actual  position  assuming  that 
the  piece  of  ordnance  is  buried  1.0  meter  (3.3  feet).  Each  of  the  86  detected  locations  has  two  entries  as 
endpoints  in  the  target  database;  one  assumes  a  burial  depth  of  0.1  meter  (0.3  feet),  and  the  other  assumes  a 
1 .0-meter  (3.3-foot)  burial  depth  with  a  refractive  index  of  5.  Although  SRI  requested  to  be  scored  based  on 
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these  pairs  of  declarations,  the  contract  required  the  demonstrator  to  provide  exact  target  locations.  SRI  was 
scored  in  a  similar  manner  as  all  other  demonstrators  consistent  with  the  data  reporting  requirements.  SRI 
selected  a  1. 0-meter  (3.3-foot)  burial  depth  for  the  database  because  it  was  unlikely  that  any  except  the 
largest  items  would  be  detected  any  more  deeply  because  of  the  wet  soil  conditions.  According  to  SRI,  the 
detected  item  should  lie  on  a  straight  line  between  these  two  target  declarations.  Because  the  critical  radius 
used  for  airborne  sensors  was  5.0  meters  (16.4  feet),  and  taking  the  average  distance  between  these  two 
depths  into  account,  one  of  these  two  points  should  encompass  a  baseline  target  if  it  was  actually  detected. 

SRI’s  performance  is  shown  in  Table  5.3.3-1,  which  presents  both  detection  and  localization  statistics.  SRI 
did  not  provide  type,  size,  or  class  information  so  classification  statistics  were  not  computed.  The  baseline 
for  computing  detection  performance  included  both  ordnance  and  nonordnance  items.  SRI  was  scored  on 
98.6  percent  of  the  total  area.  The  detection  ratio  of  0.01  (cited  as  PD  ordnance  in  Table  5.3.3-1)  reflects  the 
number  of  targets  detected  by  SRI  compared  to  the  total  number  of  baseline  ordnance  targets  in  die  area  they 
covered.  This  detection  probability  is  actually  lower  than  the  probability  of  detection  arising  from  random 
declarations  due  to  sensor  noise  and  other  factors  (Prandom  =  0.02).  If  SRI  was  scored  in  the  manner 
requested,  there  would  be  no  effect  on  the  PD  values;  however,  the  probability  of  false  alarms,  false  alarm 
rate,  and  Pm(lom  would  be  cut  in  half.  SRI  had  a  FAR  of  2.6  per  hectare. 

As  part  of  Phase  I,  SRI  recorded  a  PD  of  0.02  for  ordnance  items.  The  FAR  was  3.87  per  hectare.  A 
comparison  of  Phase  I  and  Phase  II  detection  performance  shows  that  SRI  obtained  a  slightly  lower  PD  value 
for  Phase  II  detection  performance  data.  SRI’s  FAR  improved  slightly,  however,  from  3.87  in  Phase  I  to  2.6 
in  Phase  II.  SRI’s  results  are  comparable  to  the  results  of  other  GPR  systems  from  Phase  I  and  Phase  II.  In 
addition,  SRI’s  results  are  comparable  to  those  of  other  airborne  demonstrators  as  they  showed  little  or  no 
capability  to  detect  ordnance. 

For  most  demonstrators,  the  probability  of  ordnance  detection  depends  on  both  the  size  and  depth  of  the 
buried  ordnance  item.  Figure  5.3.3-3,  which  is  a  scatter  plot  showing  detection  performance  as  a  function  of 
size  versus  depth,  illustrates  this  relationship  for  SRI.  In  this  figure,  there  appears  to  be  no  relationship 
between  size  and  depth  as  related  to  detection  performance.  A  relationship  is  difficult  to  see  because  only 
three  target  declarations  were  correct. 
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The  localization  error  statistics  section  of  Table  5. 3 .3-1  indicates  SRI’s  ability  to  estimate  the  locations  of 
the  targets  declared.  SRI  stated  that  depth  information  could  be  provided  only  by  performing  more  complex 
analysis  than  time:  permitted  under  this  contract.  However,  in  its  proposal  dated  February  17, 1995,  SRI 
stated  that  it  could  provide  depth  information  with  the  stereoscopic  approach  of  analyzing  SAR  data 
(SRI  1995a). 
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TABLE  533-1 

PERFORMANCE  STATISTICS  FOR  SRI 


Detection  Statistics 


Number  Baseline 

Number  Matched 

Pd*  1  . J 

Ordnance 

201 

3 

0.01  1 

Nonordnance 

20 

0 

0 

Total 

221 

3 

0.01 

P 

x  random 

0.02 

Number  False  Alarms 

83 

False  Alarm  Rate 

2.6/hectare 

|  False  Alarm  Ratio 

27.7 

II  Probability  False  Alarms 

0.0205 

Localization  Statistics 


1 — - - — — 

Mean  (m)b 

Std.  Dev.c(m)  ( 

8  Position  (x,y)  J 

dx 

1.09 

2.56 

dy 

0.31 

3.38 

Radial 

3.49 

1  DeP*  (z)  _ _ _ _ _ 1 

dz*1 

-2.38 

0.58 

idz 

2.42 

Identification  and  Classification  Statistics 


||  Number  Baseline 

Number  Detected 

Pd* 

Number  Correct 

■H 

1  Type 

Ordnance 

245 

3 

0.01 

0 

0 

Nonordnance 

27 

0 

0 

0 

NAg 

Size 

Large 

41 

3 

0.07 

0 

0 

Medium 

37 

0 

0 

0 

NA 

Small 

165 

0 

0 

0 

NA 

II  Class 

Bomb 

23 

3 

0.13 

0 

0 

Projectile 

73 

0 

0 

0 

NA 

Mortar 

143 

0 

0 

0 

NA 

Cluster 

6 

0 

0 

NA 

Notes:  *  Probability  of  detection  (based  on  Group  TMA,  fence  area  excluded) 

b  Meter 

'  Standard  deviation 

d  Square  root  of  the  mean  square  depth  error 

•  Probability  of  detection  (based  on  closest  TMA,  fence  area  excluded) 
1  Probability  of  correctly  classifying  (based  on  closest  TMA) 

*  Not  applicable 
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Figure  5J3-3 
SRI  Detection  Ability 
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5.4 


MAGNETOMETER  AND  ELECTROMAGNETIC  INDUCTION 


5.4.1  Australian  Defence  Industries,  Pty.  Ltd. 

ADI  demonstrated  from  June  1 4  through  18,1 995,  at  the  1 6B-hectare  area  at  JPG.  ADI  also  participated  in 
Phase  I  of  the  UXO  ATD  program  where  they  demonstrated  magnetometer  sensor  systems  only.  During 
Phase  n,  ADI  used  two  separate  man-portable  sensor  systems  for  its  demonstration:  magnetometer  and 
electromagnetic  induction.  The  following  sections  describe  the  ADI  electromagnetic  induction  technology, 
assess  ADTs  demonstration,  and  analyze  ADFs  results  for  both  magnetometer  and  electromagnetic 
induction  sensor  data  combined.  For  a  complete  discussion  on  ADI’s  magnetometer  demonstration  and 
analysis  of  the  magnetometer  data,  see  Section  5.1.2. 

5.4.1.1  Technology  Description 

ADI  used  a  TM-4  magnetometer  system  and  two  EM-6 1  time-domain  electromagnetic  induction  sensors  for 
its  surveying.  Each  EM-6 1  unit  consists  of  a  transmitter-receiver  frame,  an  electronics  backpack,  and  a 
hand-held  data  recorder  (see  Figure  5.4. 1-1).  The  top  coil,  mounted  40  centimeters  (16  inches)  above  the 


Figure  5.4.1 -1  ADI  EM-61  Electromagnetic  Time-Domain 
Unit 
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bottom  coil,  is  a  receiver  only.  The  lower  coil  acts  as  both  a  transmitter  and  receiver.  The  lower  coil  is  a 
transmitter  when  the  current  is  on  and  a  receiver  when  the  current  is  off.  The  transmitter,  receiver 
electronics,  and  controls  are  contained  in  a  backpack.  The  data  recorder  is  connected  to  the  electronics  in  the 
backpack. 

The  cart-mounted  EM-61  units  were  towed  manually  by  the  ADI  field  crews.  ADI  stated  that  the  EM-61 
unit  measures  conductivity  so  that  it  can  detect  nonferrous  materials  such  as  aluminum.  The  EM-61  data  set 
is  intended  to  complement  the  TM-4  magnetometer  data  set.  The  EM-61  unit  is  also  designed  to  provide 
lateral  location  accuracy  and  discriminate  multiple  targets.  ADI  stated  that  detection  by  the  EM-6 1  unit 
should  not  be  affected  by  varying  soil  conditions  or  the  proximity  of  buildings,  fences,  and  power  lines.  The 
data  are  then  gridded  for  image  processing  and  interpretation.  The  magnetometer  and  electromagnetic 
induction  data  sets  are  fused  to  produce  a  joint  interpretation  of  both  ferrous  and  nonferrous  buried  ordnance 
(ADI  1995a). 


5.4.1.2  System  Assessment 

This  section  summarizes  the  ADI  demonstration,  based  on  observations  made  in  the  field. 

Requirements  for  Technology  Implementation 

ADI  used  seven  people  to  complete  its  demonstration.  Two  people  operated  the  TM-4  magnetometer  in  the 
field.  One  person  carried  the  sensors,  followed  by  another  person  who  carried  a  PC  tethered  to  the  sensors. 
The  remaining  five  people  operated  the  two  EM-61  units,  alternating  to  allow  for  rest  periods.  All  system 
equipment  used  for  this  demonstration  was  shipped  to  JPG  from  ADI  headquarters  in  Australia.  ADI  rented 
a  utility  vehicle  locally  for  transportation.  During  the  demonstration,  repair  parts  such  as  inner  tubes  for  the 
tires  on  the  EM-61  wheeled  carts  were  purchased  locally.  Additional  odometers  and  a  replacement  circuit 
board  for  the  EM-6 1  were  also  needed  and  were  mailed  from  the  manufacturer.  ADI  used  the  support  trailer 
to  store  equipment  and  to  recharge  batteries. 

For  grid  navigation,  ADI  used  four  magnetic  marking  chains.  The  magnetic  chains  were  placed  along  east- 
west  grid  lines.  The  survey  lines  or  transects  were  conducted  in  a  north-south  fashion.  The  EM-61  has  an 
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odometer  in  the  wheel  but  can  also  be  switched  to  time-based  manual  mode.  Each  time  the  EM-6 1  passed 
the  marking  chains,  that  particular  grid  was  “marked”  in  the  computer  memory. 

Operational  Capabilities  and  Limitations 

The  cart-mounted  EM-6 1  units  were  towed  manually  by  the  ADI  field  crews.  The  cart-mounted  EM-6 1 
units  had  difficulty  on  the  rutted,  uneven  terrain,  and  several  breakdowns  occurred  during  the  demonstration. 
The  most  frequent  breakdowns  included  flat  tires  and  broken  odometer.  A  replacement  circuit  board  was 
also  required  for  one  EM-61  unit  during  the  demonstration. 

5.4.13  Measured  Performance 

ADI  covered  the  entire  16B-hectare  area  with  the  TM-4  magnetometer  in  about  32  hours.  Because  of  several 
breakdowns,  the  EM-61  units  covered  about  86  percent  of  the  16  hectares  in  40  hours.  ADI’s  results  were 
analyzed  in  two  ways.  The  TM-4  magnetometer  data  were  analyzed  separately,  and  the  magnetometer  and 
EM-61  data  were  analyzed  together  to  better  determine  the  ferrous  and  the  nonferrous  items  detected. 
Because  ADI  covered  more  than  50  percent  of  all  grid  cells,  both  sets  of  ADI  results  were  scored  as 
100  percent  coverage  (ADI  1995b).  See  Section  5.1.2.3  for  a  discussion  of  ADI’s  magnetometer 
performance. 

After  the  removal  of  the  fence  line  from  the  area  searched,  ADI  reported  598  targets  with  the  combined 
systems  (see  Figure  5.4.1-2)  within  the  16  hectares.  ADI’s  performance  with  the  combined  magnetometer 
and  electromagnetic  induction  systems  is  shown  in  Table  5.4. 1-1,  which  presents  the  following: 

(1)  detection,  (2)  localization,  and  (3)  classification  statistics  with  respect  to  type,  size,  and  class. 

The  baseline  for  computing  detection  performance  for  Phase  II  included  both  ordnance  and  nonordnance 
items.  The  detection  ratio  of  0.65  (cited  as  PD  ordnance  in  Table  5.4. 1-1)  reflects  the  number  of  ordnance 
targets  detected  by  ADI  with  the  magnetometer  and  electromagnetic  induction  sensor  systems  combined. 
These  PDs  are  significantly  different  than  the  probability  of  detection  resulting  from  random  declarations, 
shown  as  0.05  (PmAoJ  for  the  combined  data  set.  ADI  had  a  FAR  of  34.5  per  hectare  for  the  combined  data 


For  most  demonstrators,  the  probability  of  ordnance  detection  depends  on  both  the  size  and  the  depth  of  the 
buried  ordnance  item.  Figure  5.4.1-3  is  a  scatter  plot  showing  detection  performance  as  a  function  of  size 
versus  depth  illustrating  this  relationship.  ADI  was  more  successful  at  detecting  medium  and  large  targets 
than  smaller  targets.  Small  targets  (less  than  1 00-mm  diameter)  were  difficult  for  ADI  to  detect  at  most 
depths. 

The  localization  error  statistics  section  of  Table  5.4. 1-1  indicates  ADI’s  ability  to  estimate  the  location  of  the 
targets  declared.  ADI  reported  target  depths  between  0.39  meter  (1.28  feet)  above  surface  to  6.08  meters 
(19.95  feet)  below  ground  surface.  ADI  provided  type,  size,  and  class  information,  as  shown  in  the  tables. 
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Figure  5.4.1 -2 
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TABLE  5.4.1-1 

PERFORMANCE  STATISTICS  FOR  ADI  (MAGNETOMETER  AND  EM  INDUCTION) 


Detection  Statistics 


j 

Number  Baseline 

Number  Matched 

p. 

|  Ordnance 

92 

60 

0.65 

Nonordnance 

8 

6 

0.75 

Total 

100 

66 

0.66 

■^random 

0.05  H 

Number  False  Alarms 

561  1 

False  Alarm  Rate 

34.5/hectare 

False  Alarm  Ratio 

9.35 

Probability  False  Alarms 

0.0433 

Localization  Statistics 


1 - 

Mean  (m)b 

Std.  Dev.c(m)  | 

1  Position  (x,y)  U 

dx 

-0.07 

061  1 

dy 

-0.20 

0.59 

Radial 

0.74 

I  Depth  (z)  _ _ | 

dz4 

0.12 

0.68 

dzl 

0.68 

Identification  and  Classification  Statistics 


JP"  Number  Baseline 

Number  Detected 

mm 

Number  Correct  \ 

ra 

Type 

I  Ordnance 

125 

62 

0.50 

45 

0.73 

Nonordnance 

14 

9 

0.64 

1 

0.11 

Size 

Large 

31 

20 

0.65 

13 

0.65 

Medium 

23 

15 

0.65 

8 

0.53 

Small 

69 

26 

0.38 

24 

0.92 

Class 

Bomb 

15 

13 

0.87 

10 

0.77 

Projectile 

51 

26 

0.51 

10 

0.38 

|  Mortar 

53 

20 

0.38 

7 

0.35 

!  Cluster 

6 

3 

0.50 

0 

0 

Notes:  *  Probability  of  detection  (based  on  Group  TM  A,  fence  area  excluded) 

b  Meter 

c  Standard  deviation 

d  Square  root  of  the  mean  square  depth  error 

*  Probability  of  detection  (based  on  closest  TM  A,  fence  area  excluded) 
1  Probability  of  correctly  classifying  (based  on  closest  TMA) 
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5.4.2 


Geo-Centers,  Inc. 


Geo-Centers,  Inc.  (Geo-Centers),  demonstrated  from  August  9  through  13, 1995,  at  the  16B-hectare  area  at 
JPG.  Geo-Centers  also  participated  in  Phase  I  of  the  UXO  ATD  program. 

5.4.2.1  Technology  Description 


Geo-Centers  used  a  ground-based,  vehicle-towed  system  and  two  man-portable  systems  to  complete  its 
demonstration  at  JPG.  The  vehicle-towed  system  is  called  the  surface-towed  ordnance  locator  system 
(STOLS),  which  consists  of  a  tow  vehicle  and  tow  platform  connected  by  a  tow  bar  (see  Figure  5.4.2-1). 
The  tow  platform  carries  eight  total-field  cesium  vapor  magnetometers  positioned  0.5  meter  (1.6  feet)  apart 
along  a  boom.  The  magnetometers  can  be  adjusted  from  15  to  20  centimeters  (6  to  8  inches)  above  the 
ground.  The  eight  sensors  are  arranged  in  pairs  and  used  as  four  sets  of  total-field  gradiometers.  This  total- 
field  magnetometer  and  gradiometer  configuration  is  used  to  differentiate  small  targets  from  large  targets  in 


Figure  5.4.2-1  Geo-Centers,  Inc.,  Surface  Towed  Ordnance  Locator  System 
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close  proximity.  Also  included  on  the  platform  is  the  sensor  control  computer  that  records  the  magnetometer 
data  and  a  GPS  dome  antenna.  High  resolution  magnetic  maps  of  the  surveyed  area  are  produced  from  the 
data  collected. 

The  tow  vehicle  is  an  off-road  two-wheel  drive  single  passenger  machine  composed  of  nonmagnetic 
materials.  Geo-Centers  stated  in  its  proposal  that  the  STOLS  is  able  to  travel  in  excess  of  32  kilometers  per 
hour  (20  miles  per  hour)  over  rough  terrain  without  adversely  impacting  the  quality  of  data  collected.  The 
system  is  transported  in  a  fully  equipped  trailer,  which  becomes  an  on-site  command  center.  This  trailer  is 
self  powered  and  used  for  data  processing  and  equipment  maintenance  and  storage.  Differential  GPS  is  used 
for  navigational  precision  to  within  0.10  meters  (4  inches)  for  synchronizing  and  positioning  of  the 
multisensor  data  (Geo-Centers  1995). 


Figure  5.42-2  Geo-Centers  EM-61  Electromagnetic  Unit 


At  JPG,  two  portable,  pulsed,  induction  systems  were  used  to  cover  areas  that  could  not  be  accessed  by 
STOLS.  Two  different  types  of  electromagnetic  sensors  were  used,  the  EM-61  (see  Figure  5 .4.2-2)  and  the 
Schiebel  electromagnetic  sensor  (see  Figure  5A2-3),  both  small  coil  systems.  Two  0.5-meter  (1.5-foot)  cod 
EM-6 1  units  were  joined  together  into  one  sensor  system.  The  Schiebel  electromagnetic  sensor  has  an  array 
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of  eight  25-centimeter  (10-inch)  coil  sets  and  is  a  prototype  system.  The  linear  array  of  small,  pulsed 
induction  coils,  is  designed  to  resolve  and  detect  all  small,  shallow  metal  objects.  The  array  is  driven 
sequentially  across  each  coil  to  form  an  array  scan.  Every  eight  of  these  array  scans  are  averaged  to  form  an 
array  output  every  64  milliseconds.  The  output  is  connected  to  a  PC  for  setup,  display,  storage,  and  analysis. 


Figure  5. 4. 2-3  Geo-Centers  Schiebel  EM  Sensors 


5.4.2.2  System  Assessment 

This  section  summarizes  the  Geo-Centers  demonstration,  based  on  observations  made  in  the  field. 
Requirements  for  Technology  Implementation 

Geo-Centers  used  nine  people  to  complete  its  demonstration.  Three  people  drove  the  STOLS  vehicle  and 
operated  the  man-portable  systems,  alternating  these  responsibilities.  One  person  was  responsible  for  the 
system  electronics.  One  person  handled  data  analysis  and  the  generation  of  the  magnetic  maps.  The  survey 
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was  supervised  by  one  person.  The  remaining  three  people  were  on  site  for  only  part  of  the  survey  to 
observe  the  operations  and  assist  as  needed. 

All  system  equipment  used  for  this  demonstration  was  transported  to  JPG  by  Geo-Centers  in  a 
self-contained  trailer.  Equipment  required  in  addition  to  the  actual  STOLS  system  included  one  laptop  PC 
for  data  downloading,  one  magnetic  base  station,  and  one  GPS  base  station.  The  GPS  also  included  several 
radio-  frequency  (RF)  repeater  modems  needed  to  broadcast  the  GPS  corrections  to  the  GPS  receiver  on  the 
STOLS.  Support  equipment  used  by  Geo-Centers  included  several  vehicles  used  to  transport  personnel. 
System  batteries  were  recharged  in  the  Geo-Centers  command  station.  Data  analysis  also  took  place  there. 

Geo-Centers  established  a  reference  magnetometer  outside  of  the  16B-hectare  area,  in  an  area  that  was 
magnetically  clean.  These  data  were  recorded  at  a  1 0  Hz  rate,  downloaded  to  the  processing  computer  after 
the  survey  completion,  and  subtracted  from  the  magnetometer  data  recorded  on  the  sensor  platform. 

Operational  Capabilities  and  Limitations 

The  STOLS  used  by  Geo-Centers  required  little  maintenance  throughout  the  demonstration,  although  a  rear 
shock  absorber  was  replaced  on  the  tow  vehicle  causing  a  minor  delay.  Other  minor  equipment  problems 
included  a  flat  tire  on  the  platform  and  the  reattachment  of  the  skid  plate  that  protects  the  sensors.  The 
STOLS  vehicle  became  stuck  in  the  mud  twice,  but  it  was  quickly  freed.  In  wet  areas,  Geo-Centers  added 
extra  tires  to  the  tow  vehicle,  which  improved  traction.  The  STOLS  vehicle  traversed  the  terrain  and 
maneuvered  around  trees  with  few  problems. 

The  two  man -portable  electromagnetic  systems  were  only  used  in  a  few  select  areas.  The  dual  EM-61 
system  developed  a  broken  frame  and  was  modified  several  times  during  the  demonstration.  The  Schiebel 
electromagnetic  sensor  was  only  used  in  two  small  areas;  setup  and  operation  appeared  to  be  time- 
consuming. 


According  to  Geo-Centers,  STOLS  magnetometer  data  are  acquired  at  a  rate  of  about  100,000  magnetometer 
points  per  acre.  This  rate  is  based  on  survey  speeds  of  about  5.6  kilometers  (3.5  miles)  per  hour. 
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5.4.2J 


Measured  Performance 


Geo-Centers  surveyed  the  entire  16B-hectare  area  with  its  systems  in  the  allotted  40  hours.  Geo-Centers 
reported  1,409  targets  within  the  16B-hectare  area  (see  Figure  5. 4.2-4)  (Geo-Centers  1995).  Geo-Centers 
performance  is  shown  in  Table  5.4.2-1,  which  presents  the  following:  (1)  detection,  (2)  localization,  and 
(3)  classification  statistics  with  respect  to  type,  size,  and  class. 

The  baseline  for  computing  detection  performance  included  both  ordnance  and  nonordnance  items.  The 
detection  ratio  of  0.72  reflects  the  number  of  targets  detected  by  Geo-Centers  compared  to  the  total  number 
of  baseline  ordnance  targets  in  the  16B -hectare  area,  with  the  fence  line  area  removed.  This  detection 
probability  is  significantly  different  than  the  probability  of  detection  arising  from  random  declarations  due  to 
sensor  noise  and  other  factors  (P™^  =  0.10).  Geo-Centers  had  a  FAR  of  84.0  per  hectare. 

In  Phase  I,  Geo-Centers  had  a  PD  of  0.53  for  ordnance  items.  The  FAR  was  14.3  per  hectare.  A  comparison 
of  Phase  I  and  Phase  II  detection  performance  shows  that  Geo-Centers  obtained  higher  PD  values  for  Phase 
n,  but  higher  FAR  values  as  well. 

For  most  demonstrators,  the  probability  of  ordnance  detection  depends  on  both  the  size  and  the  depth  of  the 
buried  ordnance  item.  Figure  5.4.2-5,  which  is  a  scatter  plot  showing  detection  performance  as  a  function  of 
size  versus  depth,  illustrates  this  relationship  for  Geo-Centers.  Geo-Centers  was  successful  at  detecting 
medium  and  large  targets  at  all  depths,  but  it  had  difficulty  detecting  smaller  targets  (less  than  100-mm 
diameter)  at  most  depths. 


The  localization  error  statistics  section  of  Table  5. 4. 2-1  indicates  Geo-Centers  ability  to  estimate  the 
location  of  the  targets  declared.  Geo-Centers  reported  target  depths  between  0  and  7.8  meters  (0  and 
25.6  feet)  below  ground  surface.  Geo-Centers  declared  all  target  detections  as  ordnance.  Geo-Centers 
provided  size  and  class  information. 
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TABLE  5.4.2-1 

PERFORMANCE  STATISTICS  FOR  GEO-CENTERS 


Detection  Statistics 


Number  Baseline 

Number  Matched 

Pd* 

Ordnance 

92 

66 

0.72 

Nonordnance 

8 

8 

0.75 

Total 

100 

72 

0.72 

P 

A  random 

0.10  1 

Number  False  Alarms 

1,366  1 

|  False  Alarm  Rate 

84.0/hectare  | 

H  False  Alarm  Ratio 

20.7 

U  Probability  False  Alarms 

0.1055 

Localization  Statistics 


1 

Mean  (m)b 

Std.  Dev.£(m)  || 

|  Position  (x,y)  H 

dx 

0.02 

0.63  || 

I  dy 

-0.27 

0.67  I 

1  Radial 

0.81 

|  Depth  (z)  _ | 

dzd 

0.26 

0.84 

I  . dzl .  - . . 

0.88 

Identification  and  Classification  Statistics 


i _ 

Number  Baseline 

Number  Detected  |  PDe  |  Number  Correct  Pcf  || 

II  Type  H 

|  Ordnance 

125 

68 

68 

1 

I  Nonordnance 

14 

11 

0.79 

0 

0 

1 size  _ _ 1 

|  Large 

31 

24 

0.77 

16 

0.67 

|  Medium 

23 

15 

0.65 

12 

0.80 

1  Small 

69 

28 

0.41 

24 

0.86 

H  Class  fl 

|  Bomb 

15 

15 

1.00 

14 

0.93 

1  Projectile 

51 

29 

0.57 

22 

0.76 

1  Mortar 

53 

20 

0.38 

0 

0 

1  Cluster 

6 

4 

0.67 

0 

0 

Notes:  *  Probability  of  detection  (based  on  Group  TMA,  fence  area  excluded) 

b  Meter 

‘  Standard  deviation 

*  Square  root  of  the  mean  square  depth  error 

*  Probability  of  detection  (based  on  closest  TMA,  fence  area  excluded) 
'Probability  of  correctly  classifying  (based  on  closest  TMA) 
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Depth  (meters) 


Figure  5.4.2-5 

Geo-Centers  Detection  Ability 
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Geophex  Ltd. 


Geophex  Ltd.  (Geophex)  demonstrated  from  August  23  through  27, 1995,  at  the  16A-hectare  area  at  JPG. 
Geophex  used  two  man-portable  systems  for  its  demonstration. 

5.43.1  Technology  Description 

Geophex  used  four  Geometries  G-858  total-field  magnetometers  and  three  Geophex  electromagnetic  GEM-2 
units  during  the  demonstration.  The  G-858  (see  Figure  5.4.3-1)  consists  of  a  three-axis  fluxgate  sensor, 
custom  processing  electronics,  and  computer  with  operating  software.  According  to  Geophex,  the  G-858 
provides  total-field  readings  at  a  rate  of  30.7  samples  per  second  with  resolution  greater  than  1  nanotesla. 
Each  G-858  has  two  cesium  magnetometer  sensors  horizontally  separated  by  0.76  meter  (2.5  feet)  and 
oriented  at  a  45-degree  angle  relative  to  the  ground  for  optimum  detection  ability.  The  operator  held  these 
sensors  about  0.5  meter  (1.5  feet)  above  the  ground  surface.  The  data  from  each  sensor  are  recorded 
individually  on  the  G-858  along  with  operator-marked  fiducials  and  time.  Each  G-858  contains  a  data 
control,  acquisition,  and  field  display  unit  (Geophex  1995a). 


Figure  5.43-1  Geophex  Ltd.  G-858  Magnetometer 
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Figure  5.43-2  Geophex  Ltd.  GEM-2  Electromagnetic  Unit 


The  G-858  is  used  in  tandem  with  three  GEM-2  digital,  frequency-domain  units  (see  Figure  5.43-2),  which 
sense  both  ferrous  and  nonferrous  materials.  The  GEM-2  units  weigh  about  2.0  kilograms  (4.4  pounds). 
The  GEM-2  operated  simultaneously  at  two  frequencies,  1,350  Hz  and  7 ,290  Hz,  using  a  pulse-width 
frequency  technique.  Data  are  stored  in  solid-state  memory  and  transferred  to  a  laptop  computer  in  the  field. 
The  G-858  measures  total  magnetic  field  and  changes  caused  by  ferrous  material.  The  GEM-2  measures 
changes  in  conductivity  to  detect  ferrous  and  nonferrous  materials.  Geophex  assimilated  magnetic  and 
electromagnetic  data  for  data  analysis  and  processing  using  a  nonlinear  inversion  algorithm,  developed  and 
written  by  Geophex  personnel  (Geophex  1995a). 

In  addition  to  the  G-856  magnetometers  and  GEM-2  electromagnetic  units,  Geophex  used  two  Schoenstedt 
metal  detectors,  which  were  not  included  in  its  proposal.  These  metal  detectors  were  used  to  verify  the 
existence  of  subsurface  anomalies  and  the  location  of  metal  fencing  in  the  tree  lines. 
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5.432 


System  Assessment 


This  section  summarizes  the  demonstration  by  Geophex,  based  on  observations  made  in  the  field. 

Requirements  for  Technology  Implementation 

Geophex  used  nine  people  to  complete  its  demonstration.  Geophex  personnel  alternated  as  survey  crews  and 
support  personnel  to  provide  rest  periods.  Because  the  G-858  and  the  GEM-2  were  operated  by  a  single 
person,  logistical  limitations  are  a  result  of  human  endurance  and  ability.  All  system  equipment  used  for  this 
demonstration  was  driven  to  JPG  by  Geophex  personnel.  Geophex  had  an  ample  supply  of  spare  batteries 
for  its  survey  equipment.  No  additional  equipment  was  acquired  locally. 

Geophex  originally  brought  two  G-858  magnetometers  for  this  demonstration.  However,  on  the  first  day  of 
the  demonstration,  one  magnetometer  malfunctioned.  Geophex  received  two  additional  G-858 
magnetometers  via  overnight  delivery  to  complete  the  last  3  days  of  the  demonstration. 

Before  conducting  the  survey,  two  Geophex  personnel  cut  and  cleared  thick  vegetation  and  low  branches 
from  the  site  to  improve  access.  The  GEM-2  was  carried  through  densely  wooded  areas  much  more  easily 
than  the  G-858  magnetometers.  Navigation  across  the  grid  was  completed  using  the  grid  coordinate  system 
on  the  1 6A-hectare  area  and  was  aided  by  spray  paint  dashes  at  1 .5-meter  (4.9-foot)  intervals  in  the  north- 
south  direction.  Geophex  personnel  marked  the  grid  in  30-  (north-south)  by  60-  (east-west)  meter  (100-  by 
200-foot)  grids  prior  to  surveying.  Surveying  was  completed  in  east-west  fashion. 

Operational  Capabilities  and  Limitations 

Both  the  G-858  and  the  GEM-2  data  were  input  into  a  cart-mounted  laptop  computer  powered  by  a  12-volt 
car  battery.  Data  were  interpreted  using  two  modeling  programs  developed  by  Geophex.  The  data  were  then 
gridded  for  image  processing  and  interpretation.  After  data  review,  Geophex  determined  whether  sections  of 
the  site  needed  to  be  resurveyed.  Geophex  resurveyed  about  2  hectares  (5  acres)  with  G-858  magnetometers. 

Data  downloading  from  the  dataloggers  was  completed  in  the  field.  Data  downloading  required  5  to 
1 0  minutes;  however,  the  survey  equipment  and  crew  remained  in  the  field  to  lessen  the  delay  from 
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downloading.  The  GEM-2  dataloggers  were  capable  of  storing  data  from  about  0.8  hectare  (2.0  acres) 
before  downloading  was  necessary.  The  G-858  magnetometers  were  downloaded  about  every  1 .6  hectares 
(4.0  acres). 

5.433  Measured  Performance 

Geophex  covered  all  of  the  1 6  A-hectare  area  with  both  the  G-858  and  the  GEM-2  in  about  37  hours  (of 
40  hours  allotted).  Geophex  assimilated  magnetic  (G-858)  and  electromagnetic  (GEM-2)  data  for  analysis 
and  processing  to  find  the  spatial  position  and  depth  of  the  larger  targets.  The  interpretations  of  target  size, 
depth,  and  confidence  levels  are  based  on  the  final,  combined  data  set  (Geophex  1995b).  Geophex  reported 
398  targets  within  the  16  hectares  (see  Figure  5.43-3).  Geophex’s  performance  is  shown  in  Table  5.43-1, 
which  presents  the  following:  (1)  detection,  (2)  localization,  and  (3)  classification  statistics  with  respect  to 
type  and  size.  Geophex  did  not  provide  class  information. 

The  baseline  used  to  compute  detection  performance  included  both  ordnance  and  nonordnance  items.  The 
detection  ratio  of  0.71  (cited  as  PD  ordnance  in  Table  5.43-1)  reflects  the  number  of  targets  detected  by 
Geophex  as  compared  to  the  total  number  of  baseline  ordnance  targets  in  the  area  covered.  This  detection 
probability  is  significantly  different  than  the  probability  of  detection  arising  from  random  declarations  due 
to  sensor  noise  and  other  factors  (P^om =  0.03).  Geophex  had  a  FAR  of  19.7  per  hectare. 

For  most  demonstrators,  probability  of  ordnance  detection  depends  on  both  the  size  and  the  depth  of  the 
buried  ordnance  item.  Figure  5.43-4,  which  is  a  scatter  plot  showing  detection  performance  as  a  function  of 
size  versus  depth,  illustrates  this  relationship  for  Geophex.  Geophex  was  more  successful  at  detecting 
medium  and  large  targets  than  smaller  targets. 

The  localization  error  statistics  section  of  Table  5.43-1  indicates  Geophex’s  ability  to  estimate  the  location 
of  the  targets  declared.  Geophex  reported  target  depths  between  0.03  and  5.15  meters  (0.10  and  16.90  feet) 
below  ground  surface.  Table  5.43-1  also  shows  Geophex’s  type  and  size  capabilities;  no  class  information 
was  provided. 
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Northing  (meters) 


Figure  5.43-3 

Geophex  Target  Declarations 
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TABLE  5.43-1 

PERFORMANCE  STATISTICS  FOR  GEOPHEX 


Detection  Statistics 


1— - 

Number  Baseline 

Number  Matched 

Pd* 

1  Ordnance 

127 

90  _j 

0.71 

|  Nonordnance 

41 

25 

0.61 

Total 

168 

115 

0.69 

P 

A  random  . 

0.03 

Number  False  Alarms 

307 

|  False  Alarm  Rate 

19.7/hectare 

|  False  Alarm  Ratio 

3.41 

II  Probability  False  Alarms 

0.0248 

Localization  Statistics 


||  Mean  (m)b 

Std.  Dev.c(m)  1 

I  Position  (x.y)  _ _ _ _ 

I  *** 

-0.04 

0.83 

dy 

0.01 

h  0.62 

Radial 

0.91 

1  Depth  (z)  _ _ _ 8 

At6 

0.13 

0.61 

dzl 

0.62 

Identification  and  Classification  Statistics 


i 

Number  Baseline 

Number  Detected 

mm 

Number  Correct 

■H 

1  Type 

Ordnance 

158 

90 

0.57 

86 

0.96 

Nonordnance 

65 

31 

0.48 

7 

0.23 

Size 

Large 

35 

25 

0.71 

22 

Medium 

53 

37 

0.70 

14 

0.38 

Small 

69 

28 

0.41 

12 

0.43 

Class 

Bomb 

21 

17 

0.81 

0 

0 

Projectile 

69 

40 

0.58 

0 

0 

Mortar 

59 

28 

0.47 

0 

0 

Cluster 

9 

5 

0.56 

0 

0 

Notes:  *  Probability  of  detection  (based  on  Group  TMA,  fence  area  excluded) 

b  Meter 

e  Standard  deviation 

d  Square  root  of  the  mean  square  depth  error 

•  Probability  of  detection  (based  on  closest  TMA,  fence  area  excluded) 
'  Probability  of  correctly  classifying  (based  on  closest  TMA) 
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Figure  5.43-4 
Geophex  Detection  Ability 
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ELECTROMAGNETIC  INDUCTION  AND  GROUND  PENETRATING  RADAR 
SENSOR  SYSTEMS 


5.5.1  Coleman  Research  Corporation 

Coleman  Research  Corporation  (Coleman)  demonstrated  from  June  14  through  18,  1995,  at  the  16B-hectare 
area  at  JPG.  Coleman  also  participated  in  Phase  I  of  the  UXO  ATD  program. 

5.5.1.1  Technology  Description 

Coleman  used  the  Towed  Multi-Sensor  Array  System  (ToMAS)  for  most  of  its  demonstration  at  JPG  (see 
Figure  5.5. 1-1).  According  to  Coleman,  ToMAS  is  a  multisensor  detection  system  capable  of  detecting 
metallic  and  nonmetalhc  objects  and  identifying  soil  characteristic  changes.  The  ToMAS  consists  of  two 
sensor  arrays,  including  a  five-element  array  of  GPR  and  a  three-element  array  of  time-domain 


Figure  5.5.1-1  Coleman  Research  Corporation  Towed  Multi-Sensor  Array  System 
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electromagnetic  (TDEM)  sensors  (also  called  EM-61  units).  The  GPR  uses  a  frequency-stepped,  wide  band 
radar  capable  of  detecting  metallic  and  nonmetallic  objects  up  to  5  meters  (16  feet)  deep.  The  TDEM  array 
detects  ferrous  and  nonferrous  metal  objects  up  to  4  meters  (13  feet)  deep.  The  sensors  cover  a  2. 44-meter 
(8.00-foot)  scan  width.  ToMAS  is  designed  to  achieve  a  scan  rate  of  more  than  0.8  hectare  (2.0  acres)  per 
horn.  Data  are  collected  and  stored  on  two  distinct  media  during  the  scanning  process  and  are  later  recalled 
for  data  post-processing.  Coleman  uses  a  data  fusion  workstation  to  process  and  combine  the  data  for  two- 
and  three-dimensional  displays  (Coleman  1995a). 

The  GPR  sensor  array  portion  of  ToMAS  is  a  wideband,  coherent  stepped  frequency  radar  (100  to 
1 ,000  MHz)  with  receiver  and  transmitter  antennas.  The  use  of  a  frequency-stepped  radar  provides  greater 
sensitivity  and  greater  instantaneous  dynamic  range.  The  current  configuration  uses  two  transmitter  and 
three  receiver  spiral  antennas.  The  number  of  receiver  and  transmitter  antennas  can  be  varied  as  dictated  by 
scan  rate  requirements  and  scan  width  limitations. 

The  TDEM  sensor  array  used  in  ToMLAS  consists  of  three  EM-6 1  high-sensitivity  metal  detectors  (for  a 
complete  description  of  the  EM-61,  see  Section  5. 1.1.1).  The  EM-61  unit  generates  electromagnetic  pulses 
at  15  Hz,  and  performs  measurements  during  the  off-time  between  pulses.  The  EM-61  system  pauses  until 
the  response  from  the  conductive  earth  dissipates,  then  measures  the  prolonged  metal  response  from  buried 
metal  objects. 

Location  scanning  and  grid  layout  were  achieved  by  combining  differential  GPS  and  linear  position  encoder 
wheel  data.  The  key  components  of  the  differential  GPS  and  data  capture  system  are  the  differential  base 
station,  the  remote  GPS  station,  the  RF  communication  link,  a  linear  position  encoder  wheel,  and  the  data 
processing  computer.  The  base  station  computes  its  current  GPS  location  and  compares  it  to  stored 
reference  data  The  differences  (on  a  satellite-by-satellite  basis)  are  transmitted  to  the  remote  GPS  over  the 
RF  modem  and  are  used  to  correct  for  the  majority  of  the  GPS  errors.  This  allows  for  real-time  track 
accuracies  of  two  meters  on  moving  vehicles.  Better  accuracy  is  possible  with  postfiltering. 

The  second  system  used  by  Coleman  was  a  single  EM-61  that  was  used  in  areas  inaccessible  by  ToMAS. 
Coleman  used  the  grid  layout  to  navigate  the  single  EM-61  in  these  inaccessible  areas. 
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GPR  data  are  processed  using  fast,  two-dimensional  synthetic  aperture  imaging  algorithms;  the  TDEM  data 
are  processed  through  algorithms  developed  by  Coleman  for  improved  lateral  and  vertical  resolution.  A  data 
fusion  workstation  uses  synergism  between  the  sensors,  physical  models,  and  the  position  data  from  the 
differential  GPS  navigation  system  to  build  and  display  three-dimensional,  multisensor  reconstruction  of  the 
data  (Coleman  1995a). 

System  improvements  from  Phase  I  for  Colman’s  ToMAS  include:  modifications  to  the  GPR  antennae  and 
RF  receiver,  installation  of  differential  GPS  processing  algorithms;  improved  data  processing;  and  the 
addition  of  a  hand-held  EM-61  unit  for  coverage  in  areas  inaccessible  by  the  ToMAS  (Coleman  1995a). 


5.5.1.2  System  Assessment 

This  section  summarizes  the  demonstration  by  Coleman,  based  on  observations  made  in  the  field. 

Requirements  for  Technology  Implementation 

Coleman  used  five  people  to  complete  its  demonstration.  Two  people  operated  the  ToMAS,  alternating  to 
provide  rest  periods.  The  remaining  three  people  analyzed  the  data  and  operated  the  hand-held  sensor  as 
needed. 

The  ToMAS  sensors,  the  ATV,  and  the  GPS  equipment  were  driven  to  JPG  by  Coleman  personnel  in  a  panel 
truck.  Coleman  used  two  laptop  computers  to  store  and  process  the  data,  two  minivans  to  transport 
personnel  and  supplies,  and  a  panel  truck  to  store  the  assembled  equipment  in  the  field.  Coleman  marked  the 
area  surveyed  in  the  field  with  spray  paint  while  moving  in  a  helical  fashion  towards  the  inside  of  the  grids; 
the  GPS  recorded  the  positional  movement  of  ToMAS.  A  tarp  secured  with  elastic  straps  was  used  to  cover 
the  ToMAS.  Coleman  used  the  electrical  supply  provided  at  the  support  trailer  to  recharge  the  system 
batteries.  No  equipment  was  acquired  locally. 

Operational  Capabilities  and  Limitations 

The  ToMAS  can  cover  about  0.8  hectare  (2.0  acres)  per  hour  depending  on  the  terrain.  The  large  trees  and 
deep  ruts  in  part  of  the  area  were  difficult  for  the  ToMAS  to  maneuver  around.  Downloading  of  the 
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collected  data  was  required  about  every  4  hours  and  took  45  minutes  to  complete.  The  ToMAS  system 
experienced  numerous  delays  due  to  equipment  failure.  The  EM-6 1  array  of  the  ToMAS  had  several  flat 
tires  as  well  as  wheel  bearing  failures.  Coleman  also  experienced  electronic  problems  with  the  ToMAS  at 
one  point  during  the  demonstration,  and  the  EM-61  array  was  not  operational  for  some  of  the  time  as  well. 
All  of  these  mechanical  problems  were  corrected  on  site. 

5.5.1 .3  Measured  Performance 

Coleman  covered  all  of  the  16  hectares  with  its  combined  systems  during  the  allotted  40  hours.  Coleman 
reported  280  targets  within  the  1 6B-hectare  area  with  the  fence  line  area  removed  (see  Figure  5.5.1-  2) 
(Coleman  1995b).  An  analysis  of  Coleman’s  performance  is  shown  in  Table  5.5.1-1,  which  presents  the 
following:  (1)  detection,  (2)  localization,  and  (3)  classification  statistics  with  respect  to  size  and  class. 
Coleman  did  not  provide  type  information. 

The  baseline  for  computing  detection  performance  included  both  ordnance  and  nonordnance  items.  The 
detection  ratio  of  0.29  (PD  ordnance)  reflects  the  number  of  targets  detected  by  Coleman  compared  to  the 
total  number  of  baseline  ordnance  targets  in  the  16  hectares.  This  detection  probability  is  significantly 
different  from  the  probability  of  detection  arising  from  random  declarations  due  to  sensor  noise  and  other 
factors  (P^ob,  =  0.02).  Coleman  had  a  FAR  of  15.9  per  hectare.  It  is  not  known  to  what  degree  Coleman’s 
relatively  low  score  was  influenced  by  the  use  of  GPR. 

As  part  of  Phase  I,  Coleman  had  a  PD  of  0.40  for  ordnance  items.  Coleman’s  FAR  was  56.0  per  hectare.  A 
comparison  of  Phase  1  and  Phase  II  detection  performance  shows  that  Coleman  had  a  lower  PD  value  for 
Phase  II  as  compared  to  Phase  I.  Lower  detection  performance  in  Phase  II  may  have  been  caused  by  the 
increased  rainfall  in  May  and  the  affects  of  soil  moisture  on  GPR  performance.  Coleman  performed  Phase  I 
activities  in  early  August  1994  and  Phase  II  activities  in  June  1995.  However,  Coleman’s  FAR  improved  in 
Phase  n,  to  15.9  per  hectare  as  compared  to  56.0  in  Phase  I. 

For  most  demonstrators,  the  probability  of  ordnance  detection  depends  on  both  the  size  and  depth  of  the 
buried  ordnance  item.  Figure  5.5.1-3,  which  is  a  scatter  plot  showing  detection  performance  as  a  function  of 
size  versus  depth,  illustrates  this  relationship  for  Coleman.  Coleman  was  able  to  detect  most  of  the  medium 
targets,  but  had  difficulty  detecting  small  and  large  targets. 
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The  localization  error  statistics  section  of  Table  5.5. 1-1  indicates  Coleman’s  ability  to  estimate  the  locations 
of  the  targets  declared.  Coleman  reported  target  depths  between  0.1  and  3.0  meters  (0.3  and  9.8  feet)  below 
ground  surface.  Coleman  declared  all  detections  as  ordnance.  Coleman  did  provide  size  and  class 
information  as  indicated  in  the  table. 
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Figure  5.5.1-2 

Coleman  Target  Declarations 
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TABLE  5.5.1 -1 

PERFORMANCE  STATISTICS  FOR  COLEMAN 


Detection  Statistics 


Number  Baseline 

Number  Matched 

Pn* 

Ordnance 

92 

27 

0.29 

Nonordnance 

2 

8 

0.25 

Total 

100 

29 

0.29 

P 

x  random 

0.02  1 

Number  False  Alarms 

258  1 

False  Alarm  Rate 

15.9/hectare 

False  Alarm  Ratio 

9.56 

Probability  False  Alarms 

0.0199  1 

Localization  Statistics 


1 - 

Mean  (m)b 

Std.  Dev.c(m) 

1  Position  (x,y)  II 

dx 

-0.05 

1.04 

dy 

0.30 

1.02  1 

Radial 

1.41 

Depth  (z)  _ _ _ _ 

dz* 

-0.36 

0.95 

Cj 

dz 

1.00 

Indentification  and  Classification  Statistics 


- — 

Number  Baseline 

Number  Detected 

wm 

Number  Correct 

■31 

Ordnance 

125 

29 

0.23 

29 

1.00 

Nonordnance 

14 

2 

0.14 

0 

0 

||  Size 

Large 

31 

7 

0.23 

1 

0.14 

Medium 

23 

8 

0.35 

2 

0.25 

Small 

69 

13 

0.19 

12 

0.92 

||  Class 

Bomb 

15 

3 

0.2 

1 

0.33 

Projectile 

51 

15 

0.29 

13 

0.87 

Mortar 

53 

10 

0.19 

0 

0 

Cluster 

6 

1 

0.17 

0 

0 

Notes:  *  Probability  of  detection  (based  on  Group  TMA,  fence  area  excluded) 

b  Meter 

‘  Standard  deviation 

d  Square  root  of  the  mean  square  depth  error 

'  Probability  of  detection  (based  on  closest  TMA,  fence  area  excluded) 
’  Probability  of  correctly  classifying  (based  on  closest  TMA) 
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Principal  Diameter  (millimeters) 


■  Target  Detected 
□  Target  Not  Detected 
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REMEDIATION  SYSTEMS 


Two  remediation  systems  were  demonstrated  at  JPG  for  Phase  II  of  the  UXO  ATD  program.  Concept 
Engineering  Group,  Inc.,  demonstrated  a  system  called  the  soft  trencher,  and  Wright  Laboratory 
demonstrated  a  remote  excavating  vehicle  system. 

5.6.1  Concept  Engineering  Group,  Inc. 

Concept  Engineering  Group,  Inc.  (CEG),  demonstrated  from  September  6  through  8, 1995,  at  the 
16A-hectare  area  at  JPG.  CEG's  soft  trencher  was  used  to  demonstrate  remediation  of  known  ordnance. 

5.6.1.1  Technology  Description 

CEG’s  soft  trencher  is  a  mobile,  self-propelled,  platform-type  excavation  technology  ( see  Figure  5.6.1-1). 
The  soft  trencher  weighs  15,436  kilograms  (kg)  (34,030  pounds)  and  is  powered  by  a  275-horsepower 
Detroit  Diesel  D-DEC  engine,  which  drives  four  hydraulic  pumps.  All  of  the  soft  trencher’s  systems  are 
hydraulically  operated.  The  soft  trencher  functions  manually  in  the  field  and  is  not  guided  by  a  ground 
positioning  navigation  system.  The  system  is  capable  of  digging  trenches  up  to  1.8  meters  (6  feet)  wide  and 
3.0  meters  (10  feet)  deep,  although  it  can  be  modified  to  excavate  to  about  4  meters  (13  feet).  The  soft 
trencher  has  a  retractable  lift,  which  removes  soil-type  materials  while  excavating,  and  uses  a  multistage 
vacuum  and  filter  system  for  soil  excavation  and  discharge.  A  conveyor  belt  moves  soil  away  from  the 
system  and  deposits  it  at  the  back  or  side  of  the  machine  (CEG  1995a). 

The  "hood"  of  the  soft  trencher  uses  supersonic  air  jets  and  a  pneumatic  vacuum  to  transport  soil  during 
excavation.  The  soft  trencher  uses  the  high  speed  air  to  penetrate  and  dislodge  soil  without  breaking  or 
puncturing  ordnance.  The  soft  trencher  consists  of  a  digging  assembly  that  uses  the  excavation  head  to 
loosen  the  soil  with  supersonic  aiijets.  The  suction  tubes  combine  with  a  vacuum  to  aerate  materials. 

The  soft  trencher  is  operated  by  a  remote,  portable  box  equipped  with  joysticks  to  allow  the  operator  to 
control  the  excavator  from  distances  up  to  7.6  meters  (25  feet)  from  the  excavator.  The  system  also  includes 
an  operator's  seat  with  a  conventional  steering  wheel,  accelerator,  brakes,  and  mirror  to  maneuver  the 
machine  to  site  excavation  targets. 
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Figure  5.6.1 -1  Concept  Engineering  Group,  Inc. 
Soft  Trencher  System 


5.6.1.2  System  Assessment 

This  section  summarizes  the  CEG  demonstration,  based  on  observations  made  in  the  field. 

Requirements  for  Technology  Implementation 

Two  people  were  on  site  for  this  demonstration.  One  person  drove  the  vehicle  and  performed  excavation 
activities.  The  other  observed  progress  of  the  excavation  activities  near  the  target.  CEG  personnel  used  the 
site  trailer  for  telephone  use  only. 
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CEG  used  local  contractors  to  support  its  field  operations  at  JPG.  Valley  Industrial  Supply  Company  Inc.,  of 
Madison,  Indiana,  provided  both  small  and  large  mechanical  parts,  including  bolts  and  winches.  Sedam 
Construction  Company  provided  welding  support  services  to  weld  and  reweld  cracks  in  the  excavator  hood 
on  two  separate  occasions.  Bullock  Oil  Company  provided  a  300-gallon  diesel  tank  with  fuel. 

CEG  needed  a  support  vehicle  stocked  with  field  supplies  such  as  gloves,  drip  pans,  and  measuring  tapes, 
which  they  did  not  have.  These  supplies  were  all  purchased  and  provided  to  CEG.  No  other  unique  support 
was  provided  to  CEG  other  than  occasional  transport  out  to  the  field,  which  was  necessary  because  CEG  did 
not  have  a  field  vehicle. 

Operational  Capabilities  and  Limitations 

Due  to  its  large  size  [2.6  meters  (8.5  feet)  wide,  8.4  meters  (27.6  feet)  long,  and  3.5  meters  (1 1.5  feet)  high], 
the  soft  trencher  has  limited  maneuvering  ability.  To  navigate  in  off-road  conditions,  its  rubber  tires  must  be 
changed  to  a  track  mount.  The  soft  trencher  was  demonstrated  during  predominantly  dry  weather,  otherwise, 
the  vehicle  may  have  required  towing  by  a  large  support  vehicle  such  as  a  bulldozer. 

The  soft  trencher  has  two  types  of  excavation  hoods:  cylinder  rotating  and  rectangular.  The  cylinder  rotating 
hood  was  used  briefly,  but  this  hood  was  not  capable  of  cutting  through  the  hard,  dense,  silty  clays 
characteristic  of  the  glacial  soils  found  at  JPG.  The  rectangular  hood  was  more  successful  because  its 
excavation  head  has  more  air  pressure.  The  rectangular  hood  was  also  used  to  physically  move  soil  in  and 
around  the  excavation  area. 

In  the  initial  excavation  at  each  target,  high  grass  and  weeds  caused  the  intake  hood  to  plug.  Additionally, 
the  rectangular  excavation  hood  and  the  conveyor  system  also  became  clogged  in  the  field,  because  the  dense 
silty  clays  were  sometimes  wet  and  included  large  angular  chert-type  rock  fragments.  These  rock  and  soil 
formations  prevented  continuous  operations  for  10-  to  15-minute  periods.  CEG  used  a  shovel  or  a  large 
metal  rod  to  remove  clogged  soils  from  the  hood  and  conveyor  belt. 

Two  problems  occurred  with  the  soft  trencher  hydraulics.  The  first  involved  a  cracked  hydraulic  hose 
feeding  the  lower  boom  extension  cylinder.  The  hose  was  by-passed,  but  this  reduced  excavation  depths  to 
about  2  meters  (6.6  feet).  The  second  problem  involved  the  inlet  fitting  to  the  boom  lift  cylinder.  The  inlet 
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fitting  was  struck  by  the  top  head  of  the  lower  boom  extension  cylinder  and  forced  out  of  the  cylinder  head. 
This  could  only  be  repaired  by  replacing  the  cylinder  head,  which  could  not  be  done  in  the  field.  At  this 
point,  the  demonstration  was  terminated. 

5.6.1. 3  Measured  Performance 

CEG  excavated  14  targets  in  18.5  of  the  24  hours  allotted  for  its  demonstration.  Table  5.6.1-1  provides 
details  of  the  demonstration  results.  The  soft  trencher  excavated  targets  consisting  of  8 1  105-,  106-,  152-, 

155-,  and  175-millimeter  (mm)  projectiles;  8-inch  projectiles;  and  a  250-pound  bomb.  The  targets  were 
excavated  in  the  order  in  which  they  appear  in  Table  5.6. 1  - 1 .  Due  to  the  limitations  of  the  soft  trencher, 
targets  were  not  removed  from  the  excavation  before  backfilling.  The  travel  rate  to  each  target  was 
determined  from  the  straight  line  distance  between  start  and  finish  points  of  travel  and  the  time  required  for 
that  travel.  The  duration  of  the  target  excavation  includes  downtime  associated  with  repair  of  the  soft 
trencher  (CEG  1995b). 

CEG  successfully  excavated  10  of  the  14  targets.  A  target  was  considered  successfully  excavated  if  it  was 
unearthed  and  observed  entirely  by  remote  excavation.  Three  of  the  four  targets  not  successfully  excavated 
were  nonbase-line  targets  (anomalies).  After  excavation  of  these  targets,  it  was  determined  in  the  field  using 
a  Schoenstedt  metal  detector  that  no  ordnance  was  apparent  in  the  excavation.  Initial  detection  of  the  target 
could  possibly  have  been  due  to  the  magnetic  signature  of  the  soil. 
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TABLE  5.6.1-1 

CONCEPT  ENGINEERING  GROUP,  INC. 
16A-HECTARE  AREA 


Target  No. 

Target  Class 

Travel  Rate 
(km/hr) 

Depth 

(meters) 

Excavation 
Duration  (hr) 

Ordnance 

Found3 

972 

Unknown 

0.09 

0.33 

1.72 

Nob 

222 

175-mm  projo 

1.10 

0.70 

0.15 

Yes 

211 

250-lb  bomb 

0.98 

0.30 

2.22 

Yes 

1030 

106-mm  projo 

0.55 

1.0 

0.27 

Yes 

236 

81-mmprojo 

0.91 

1.15 

0.15 

Yes 

213 

8-inch  projo 

3.66 

2.44 

2.55 

Yes 

913 

Unknown 

1.19 

1.0 

0.45 

Nob 

219 

155-mm  projo 

1.14 

1.31 

0.20 

Yes 

942 

Unknown 

0.64 

1.0 

0.28 

Nob 

1  282 

8-inch  projo 

0.30 

1.4 

1.30 

Yes 

242 

155-mm  projo 

1.10 

0.90 

0.23 

Yes 

359 

105-mm  projo 

1.04 

0.37 

0.6 

Yes 

1016 

152-mm  projo 

4.39 

1.0 

0.25 

Yes 

1017 

155-mm  projo 

0.31 

1.06 

0.08 

Noc 

Average 

1.24  km/hr 

0.75  hr/hole 

Notes:  *  Yes  =  Ordnance  was  found  during  excavation  with  the  soft  trencher. 

No  =  Ordnance  was  not  found. 

b  Nonbase-line  targets  (anomaly)  investigated  as  part  of  remediation,  possibly  due  to 
magnetic  signature  in  soil 

c  Soft  trencher  breakdown;  demonstration  was  terminated 

hr  Hour 

km  Kilometer 

lb  Pound 

mm  Millimeter 

projo  Projectile 
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5. 


Wright  Laboratory  Remote  Excavation  Vehicle  System  (REVS)  was  demonstrated  from  September  22 
through  25,  1995,  at  the  16A-hectare  area  at  JPG.  Wright  Laboratory  also  participated  in  Phase  I  of  the 
UXO  ATD  program,  demonstrating  its  Autonomous  Ordnance  Excavator  (AOE).  Both  systems  were  used 
to  demonstrate  remediation  of  known  ordnance. 


5.0.1 


The  REVS  consists  of  a  36,320  kg  (80,071  pounds)  remote  excavation  vehicle  and  a  mobile  command 
station  (MCS).  Figure  5.6.2-1  shows  the  REVS  excavation  vehicle.  REVS  is  designed  as  a  robotic 
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Figure  5.6.2-1  Wright  Laboratory  Remote  Excavation  Vehicle 
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vehicle  auto  leveling,  and  target  overburden  soil  removal  were  not  demonstrated.  Only  teleoperated 
manipulation  with  an  enhanced  graphical  user  interface  was  demonstrated  (Wright  Laboratory  1995), 

The  vehicle  features  a  Caterpillar  mobile  track  system  with  two  wide  rubber  tracks  for  travel  on  paved 
surfaces.  The  mobile  track  system  allows  travel  up  to  22.5  kilometers  per  hour  (km/hr)  (14  miles  per  hour). 
The  excavator  is  powered  by  a  300-horsepower  Cummins  diesel  engine  and  equipped  with  a  5-kilowatt 
generator.  A  Caterpillar  225  boom  assembly  is  mounted  for  excavation  operations.  A  Balderson  thumb  is 
installed  on  the  bucket  for  grasping  objects.  A  real-time  GPS  and  a  Modular  Azimuth  Positioning  System 
(MAPS)  are  installed  on  the  vehicle  to  provide  precise  positioning  of  the  excavator  at  the  target  location. 
MAPS  provides  the  remote  operator  with  the  proper  orientation  of  the  vehicle  during  travel  and  excavation. 

T  mear  displacement  transducers  are  built  into  the  excavator’s  hydraulic  cylinders,  referred  to  as  actuators,  to 
determine  the  overall  position  of  the  boom,  stick,  and  bucket.  This  information  is  returned  to  the  MCS  and 
graphically  displayed  to  the  remote  operator. 


The  MCS  houses  the  remote  operator  station  to  control  teleoperation.  The  base  vehicle  for  the  MCS  is  a 
Chevrolet  Multi-Stop  van.  A  telescoping,  9.1 -meter-  (30-foot-)  high  mast  turret  camera  system  provides  a 
300-degree  view  and  maintains  proper  orientation  of  die  directional  antennae  for  an  optimal  communication 
link  with  the  vehicle.  The  REVS  is  capable  of  0.62-kilometer  ( 1  -mile)  line  of  sight  operations.  The  MCS  is 
equipped  with  a  base-station  GPS  to  provide  differential  correction  of  the  vehicle  location.  A  Sun  Sparc  20 
workstation  provides  the  graphic  user  interface  for  computer  control  of  outrigger  deployment,  on-board 
camera  selection,  and  engine  throttle.  It  also  provides  real-time  graphic  orientation  of  the  vehicle.  Before 
each  excavation,  the  latitude,  longitude,  and  depth  of  the  target  is  entered  into  the  workstation  on  board  the 
MCS.  The  MCS  requires  two  operators,  one  teleoperator  for  control  of  the  REVS  and  one  operator  for  the 
computer  interface  control.  A  joystick  controller  provides  the  teleoperator  with  multiple  levels  of  control. 

5.6.2 .2  System  Assessment 

This  section  summarizes  the  CEG  demonstration,  based  on  observations  made  in  the  field. 
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Requirements  for  Technology  Implementation 

Wright  Laboratory  used  five  people  to  conduct  its  demonstration.  Two  people  operated  the  vehicle  and 
controlled  the  computer  interface.  The  other  three  people  included  a  project  leader  and  two  design  engineers 
for  electrical  and  mechanical  problem  solving.  At  least  one  person  remained  at  the  excavation  area  to 
observe  the  vehicle  during  excavation.  As  a  precaution,  the  observer  stopped  excavation  activities  several 
times  during  the  demonstration  so  that  the  REVS  could  be  repositioned  near  the  excavation. 

Wright  Laboratory  personnel  used  the  support  trailer  for  telephone  use  only.  The  support  tent  at  the 
16A-hectare  area  was  used  to  store  equipment  and  a  support  vehicle  (a  six-wheeled  John  Deere  Gator). 


The  REVS  was  used  to  backfill  two  shallow  excavations  that  resulted  from  the  excavation  of  two  anomalies, 
Targets  No.  128  and  149.  Several  small  pieces  of  metal  were  identified  at  Target  No.  128  and  are  the 
possible  remains  of  old  farm  equipment.  Target  No.  1 49  was  possibly  identified  from  magnetic  properties  of 
the  soil.  Although  the  vehicle  is  equipped  with  a  bulldozer  blade,  the  on-board  cameras  did  not  show  an 
overview  or  positioning  of  the  vehicle  near  the  excavation,  making  teleoperation  of  backfilling  activities 
difficult.  A  local  excavation  company  was  subsequently  hired  to  backfill  the  remaining  target  excavations. 

Operational  Capabilities  and  Limitations 

Navigation  across  the  grid  was  conducted  by  the  remote  teleoperator  and  aided  by  on-board  cameras  and 
MAPS.  MAPS  provided  a  computer-graphic  aerial  view  of  the  16A-hectare  area  showing  the  location  of  the 
target  and  the  vehicle.  The  on-board  cameras  provided  the  immediate  view  for  navigation  around  obstacles. 
However,  smaller  obstacles,  such  as  grid  stakes,  could  not  be  seen  by  the  vehicle  operator  and  were  often  run 

over. 

The  vehicle  was  equipped  with  four  on-board  fixed-lens  cameras.  During  the  demonstration,  the  camera 
attached  to  the  boom  located  directly  above  the  bucket  failed  and  had  to  be  replaced  with  the  front-facing 
overview  camera  on  top  of  the  vehicle,  since  Wright  Laboratory  personnel  did  not  have  a  spare  camera. 
Several  times  throughout  the  demonstration,  the  communication  link  between  the  vehicle  and  MCS  was  lost, 
and  the  REVS  computer  had  to  be  rebooted.  Each  incident  resulted  in  about  10  to  15  minutes  of  downtime. 
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During  remediation  of  the  second  target,  Target  No.  1031,  the  vehicle  operator  grasped  the  target  with  the 
Balderson  thumb  to  remove  it  from  the  excavation.  During  removal,  the  bucket  was  closed  onto  the  thumb 
bending  the  thumb  actuator.  Wright  Laboratory  personnel  indicated  that  this  is  a  design  flaw  because  the 
thumb  actuator  should  retract  when  excessive  force  is  added  to  the  bucket.  The  thumb  was  removed  from 
the  vehicle,  and  the  actuator  was  secured  to  the  boom  with  a  chain. 

The  vehicle's  bucket  had  teeth  about  0.35-meter-  (1.1  -foot-)  long.  During  every  target  excavation,  these 
teeth  displaced  the  target  so  that  the  azimuth  and  declination  could  not  be  determined.  Very  often  the  target 
could  not  be  identified  by  the  REVS  cameras,  but  were  located  during  field  examination  using  a  Schoenstedt 
metal  detector.  REVS  was  unable  to  remotely  locate  targets  smaller  than  1 06  mm  in  the  clayey  native  soil  at 
JPG.  Larger  targets  (250-  and  500-pound  bombs)  were  also  masked  by  clay  and  could  not  be  identified  until 
a  field  investigation  was  performed  with  a  Schoenstedt  metal  detector. 

Two  problems  occurred  with  the  REVS  hydraulics.  Personnel  first  had  to  replace  an  O-ring  on  a  leaking 
fitting,  and  when  a  second  O-ring  on  the  stick  cylinder  failed,  Wright  Laboratory  personnel  did  not  have  a 
replacement  O-ring,  and  the  demonstration  was  terminated. 

5.6 .23  Measured  Performance 

Wright  Laboratory  excavated  1 1  targets  in  23.5  of  the  24  hours  allotted  for  its  demonstration.  Table  5.6.2-1 
details  the  results  of  the  REVS  demonstration.  REVS  remediated  targets  consisting  of  60-mm  mortars,  106- 
and  155-mm  projectiles,  and  250-  and  500-pound  bombs.  The  targets  were  remediated  in  the  order  in  which 
they  appear  in  Table  5.6.2-1 .  The  travel  rate  to  each  target  was  determined  from  the  straight  line  distance 
between  start  and  finish  points  of  travel  and  the  time  required  for  that  travel.  The  duration  of  the  target 
excavation  includes  downtime  associated  with  rebooting  the  computers  on  board  the  vehicle  when  necessary 
and  searching  the  excavation  with  the  Schoenstedt  metal  detector. 

REVS  remotely  excavated  5  of  the  1 1  targets.  A  target  was  considered  remotely  excavated  if  it  was 
observed  and  remediated  entirely  by  video  surveillance.  A  target  was  considered  manually  excavated  if  it 
was  located  using  a  metal  detector. 
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TABLE  5.6.2-1 
WRIGHT  LABORATORY 
16A-HECTARE  AREA 


|  Target  No. 

Target  Class 

Travel  Rate 
(km/hr) 

Depth 

(meters) 

Excavation 
Duration  (hr) 

Excavation 

Method8 

306 

250-lb  bomb 

2.1 

2.98 

1.45 

Manual 

1031 

250-lb  bomb 

2.1 

1.11 

0.67 

Remote 

1  307 

250-lb  bomb 

1.5 

2.37 

1.55 

Manual 

346b 

106-mmprojo 

1.5 

0.47 

0.08 

Remote 

350c 

60-mm  mortar 

2.9 

0.03 

0.17 

Manual 

319 

106-mmprojo 

2.7 

0.23 

0.07 

Manual 

304 

250-lb  bomb 

1.8 

1.40 

0.23 

Remote 

1033 

500-lb  bomb 

4.4 

2.80 

0.72 

Remote 

1017 

155-mmprojo 

2.9 

1.06 

0.25 

Manual 

1004 

155-mmprojo 

5.5 

0.54 

0.05 

Remote 

205 

500-lb  bomb 

3.7 

3.81 

1.03 

Manual 

Average 

2.83  km/hr 

0.57  hr/hole 

Notes:  *  Remote  =  Ordnance  was  identified  on  video  survefllance  during  excavation. 

Manual  =  Ordnance  was  found  using  a  Schoenstedt  metal  detector. 
b  Target  No.  346  was  found  while  removing  overburden  soil  for  the  excavation  of 
Target  No.  307. 

c  Adjacent  targets  No.  351  and  352  were  also  removed  during  excavation  of  Target 
No.  350  and  were  found  using  a  Schoenstedt  metal  detector. 


hr 

Hour 

km 

Kilometer 

lb 

Pound 

mm 

Millimeter 

projo 

Projectile 
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6.0  COMPARISONS  AND  CONCLUSIONS 


The  overall  objective  of  the  Phase  II  demonstration  was  to  evaluate  individual  sensor  performance  and  overall 
technology  performance  to  provide  useful  information  for  UXO  technology  end-users.  Specifically, 
demonstrators  were  to  determine  target  location;  localize  ordnance  below  the  surface;  identify  and  classify 
ordnance  with  respect  to  type,  size,  and  class  of  ordnance;  and  in  the  case  of  remediation  demonstrators, 
excavate  targets. 

6.1  PERFORMANCE  STATISTICS  SUMMARY 

Tables  6-1  and  6-2  summarize  individual  demonstrator  performance  statistics  grouped  by  the  sensor 
technology  employed.  Performance  statistics  for  the  individual  demonstrators  were  evaluated  by  detection, 
localization,  identification,  and  classification. 

Detection  Performance 

When  assessing  the  detection  performance  of  a  system,  both  the  UXO  detection  capability  and  the  number  of 
fata*,  alarms  must  be  considered.  In  Figure  6-1,  the  PD  values  for  ordnance  items  are  plotted  for  each 
demonstrator.  Figure  6-2  presents  a  similar  plot  showing  the  FAR  values  in  number  per  hectare,  for  each  of 
the  demonstrators.  These  statistics  are  combined  in  Figure  6-3,  which  plots  PD  versus  PFA.  In  this  plot, 
demonstrators  with  better  detection  performances  are  located  toward  the  upper  left  corner  of  the  plot;  those 
with  the  poorer  detection  performance  are  located  toward  the  lower  right  comer.  Two  of  the  Phase  II 
performers  had  PD  values  greater  than  0.8  and  PFA  values  less  than  0.05. 

Demonstrators  using  ground-based  magnetometer  sensors  (ADI,  Geometries,  Scintrex,  and  Vallon)  had  a 
probability  of  detection  for  ordnance  in  the  range  of  0.50  to  0.83.  The  detection  capability  of  magnetometer 
sensors  appears  to  be  based  on  the  operation  of  the  equipment  and  data  processing.  Geometries  recorded  the 
highest  PD  (0.83)  using  a  hand-held  magnetometer;  Geometries  also  employed  advanced  data  processing. 
When  results  for  the  four  ground-based  magnetometer  sensor  systems  are  compared,  it  appears  that 
Geometries’  advanced  data  processing  may  account  for  its  higher  PD  value.  The  one  airborne  demonstrator 
(Aerodat)  using  magnetometer  sensors  had  little  or  no  detection  capability  (0.02). 
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TABLE  6-1 

DEMONSTRATOR  DETECTION  AND  LOCALIZATION  BY  SENSOR  TYPE 
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Note:  NA  Not  applicable 


TABLE  6-2 

DEMONSTRATOR  IDENTIFICATION  AND  CLASSIFICATION  (TYPE,  SIZE,  AND  CLASS)  BY  SENSOR  TYPE 
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NA  Not  applicable 

1  Demonstrator  reported  all  target  declarations  as  “ordnance.” 

1  Demonstrator  did  not  provide  type,  size,  and  class  information  for  declarations  or  listed  them  as  “unknown.' 


Three  demonstrators  used  EM  induction  sensors:  GeoPotential,  Parsons,  and  Bristol.  The  range  of  PD  values 
for  EM  induction  sensors  is  0. 1 1  to  0.85.  Parsons  used  an  EM-61  and  advanced  data  processing  to  obtain  a 
PD  of  0.85,  which  was  the  highest  PD  for  all  the  systems  demonstrated. 


Three  demonstrators  used  a  combination  of  magnetometer  and  EM  induction  sensors:  Geo-Centers, 

Geophex,  and  ADI.  The  results  of  these  demonstrators  are  comparable  to  the  results  of  magnetometer  and 
EM  induction  sensors  used  alone.  There  may  be  an  advantage  in  fusing  data  from  the  two  technologies,  as 
evident  by  the  narrow  range  of  PD  values  (0.65  to  0.72)  observed  for  these  demonstrators. 

GPR  sensor  equipment  was  used  by  three  demonstrators:  AES,  SRI,  and  Kaman.  The  range  of  PD  values  for 
GPR  sensors  is  0.00  to  0.05.  GPR  sensor  system  results  are  consistent  with  results  from  Phase  I  that  showed 
these  systems  as  having  little  or  no  detection  capability.  GPR  systems  may  have  performed  poorly  due  in 
part  to  the  soils  at  JPG,  which  have  a  high  clay  content  In  addition,  weather  conditions  before  and  during  all 
three  Phase  II  GPR  demonstrations  contributed  to  a  high  soil  moisture  content,  which  typically  diminishes 
GPR  performance.  The  two  airborne  demonstrations  (AES  and  SRI)  showed  no  detection  capability  during 

Phase  I  or  II.  Coleman  demonstrated  a  combination  of  EM  induction  and  GPR  sensors.  It  is  not  known  to 

what  degree  Coleman’s  relatively  low  score  was  influenced  by  the  use  of  GPR 

FAR  values  for  the  Phase  II  demonstrators  ranged  from  0.9  to  225.9  false  alarms  per  hectare.  No  direct 
relationship  exists  between  false  alarm  rate  and  P^  although  both  are  important  parameters  in  determining 
the  best  technology  for  UXO  remediation. 

To  provide  a  single  evaluation  rating  that  incorporates  both  the  PD  and  the  false  alarm  rate,  an  optimal 
performance  value  was  computed.  This  function  is  derived  by  computing  the  distance  in  probability  space  of 
the  demonstrator’s  performance  (PD ,  PFA)  relative  to  the  optimal  system  performance  (PD  =  1.00,  PFA  =  0). 
This  value  provides  another  metric  to  evaluate  demonstrator  performance,  and  it  considers  missed  detection 
on  an  equal  basis  with  false  alarms.  Low  values  indicate  better  performance.  Optimal  performance 
evaluation  values  are  presented  in  Table  6-3  and  are  plotted  in  Figure  6-4.  Figure  6-4  shows  a  ‘natural’  break 
in  the  demonstrator  rankings  at  four  different  intervals  that  are  similar  to  the  distribution  bins  presented  in  the 
Phase  I  report  (USAEC  1995).  The  best  performers  are  shown  as  Parsons  and  Geometries,  followed  by 
Geophex,  Geo-Centers,  ADI,  and  Bristol. 
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TABLE  6-3 

DEMONSTRATOR  OPTIMAL  PERFORMANCE  EVALUATION 


Demonstrator 

Optimal 

Performance 

Evaluation 

Probability  of 
Detection 

Probability  of 
False  Alarms 

Parsons 

0.16 

0.85 

0.04 

Geometries 

0.18 

0.83 

0.03 

Geophex 

0.29 

0.71 

0.02 

Geo-Centers 

0.30 

0.72 

0.11 

ADI  (MAG  &  EM) 

0.35 

0.65 

0.04 

ADI  (MAG) 

0.37 

0.63 

0.04 

Bristol 

0.38 

0.62 

0.05 

Scintrex 

0.50 

0.50 

0.06 

Vallon 

0.51 

0.57 

0.28 

Coleman 

0.71 

0.29 

0.02 

GeoPotential 

0.89 

0.11 

0.02 

AES 

0.96 

0.05 

0.01 

Aerodat 

0.98 

0.02 

0.02 

SRI 

0.99 

0.01 

0.02 

Kaman 

1.00 

0.00 

0.01 

Localization  Performance 

Figure  6-5  shows  the  horizontal  position  (or  radial)  errors  for  the  demonstrators.  Results  for  the  ground- 
based  systems  ranged  from  0.65  to  1.30  meters.  Airborne  systems  show  horizontal  position  errors  from  2.29 
to  3.49.  The  sparse  number  of  detections  for  airborne  demonstrators  probably  affects  die  significance  of 
these  estimates.  Furthermore,  for  airborne  demonstrators,  it  is  not  clear  whether  these  location  errors  are  due 
to  random  declarations,  as  indicated  by  the  low  detection  probabilities  (see  Figure  6-1). 
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Depth  errors  for  most  of  the  ground-based  demonstrators  appear  to  be  statistically  indistinguishable  from 
zero  (at  the  95  percent  confidence  level).  However,  demonstrators  that  used  EM  sensors  (Parsons,  Bristol, 
and  GeoPotential)  appear  to  have  estimated  depths  that  are  shallower  than  the  actual  target  depth,  as 
evidenced  by  the  negative  depth  errors  in  the  individual  demonstrator  tables. 

One  explanation  for  these  results  is  that  EM  sensors  are  probably  receiving  stronger  signals  from  the  tops  of 
the  buried  ordnance  items.  The  manner  in  which  the  actual  signal  is  received  is  a  complicated  function  of  the 
figuration  of  the  sensor,  the  sensor  orientation  and  position  relative  to  the  buried  UXO,  the  structure  and 
orientation  of  the  buried  UXO,  and  the  intervening  soil:  A  simplistic  explanation  for  the  shallower  depth 
estimates  require  an  understanding  of  the  way  the  sensors  function.  Conductivity  sensors  function  by 
inducing  surface  currents  in  material  that  generate  an  EM  field;  the  strength  of  the  induced  field  is  then 
measured  by  the  sensor.  The  strength  of  the  generated  signal  decreases  rapidly  with  distance  from  the  object 

(depth).  As  a  result,  the  top  ofthe  ordnance  item  would  tend  to  generate  a  stronger  signal  than  the  bottom.  If 

this  difference  in  signal  strength  is  not  accounted  for,  a  shallower  depth  estimate  will  be  given.  In  addition, 
shallower  depth  estimates  will  become  more  exaggerated  when  the  UXO  item  is  oriented  cm  end  (nose  up  or 
down)  as  was  the  case  with  many  of  the  emplaced  UXO  items  at  the  Phase  II  areas. 


Identification  and  Classification  Performance 

Demonstrators  providing  identification  and  classification  information  were  scored  on  their  ability  to 
determine  the  type  (ordnance  or  nonordnance);  size  (large,  medium,  or  small);  and  class  (bomb,  mortar, 
projectile,  cm-  cluster)  for  detections.  A  score  was  given  for  each  possible  classification  category  (type,  size, 
and  class).  Figures  6-6  through  6-8  indicate  each  ofthe  demonstrator’s  performance  in  correctly  determining 
type,  size,  and  class,  respectively.  As  described  in  Section  5.0,  not  all  of  the  demonstrators  provided 
information  in  all  three  identification  and  classification  categories.  Table  6-3  summarizes  the  identification 
and  classification  information  provided  by  the  individual  demonstrators.  Some  demonstrators  typed  all  target 
declarations  as  ordnance;  these  demonstrators  are  indicated  by  the  “ordnance  only”  entry  in  the  table. 
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DEMONSTRATOR  IDENTIFICATION  AND  CLASSIFICATION  INFORMATION  PROVIDED 


Demonstrator 


Bristol 


Geometries 


Geophex 


GeoPotential 


Kaman 


Parsons 


Scintrex 


ADI  (MAG  &  EM) 


ADI  (MAG) 


Coleman 


Geo-Centers 


cm 


Aerodat 


AES 


SRI 


No 


(ordnance  only) 


Yes 
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Figure  6-6  shows  that  demonstrator  performance  in  determining  ordnance  objects  was  significantly  better 
than  their  respective  ability  in  determining  nonordnance  objects.  Although  there  were  significantly  more 
ordnance  than  nonordnance  items  emplaced  for  Phase  n,  the  performance  differences  shown  in  Figure  6-6  are 
statistically  significant  at  the  95  percent  confidence  level. 


Figure  6-7  shows  that  demonstrators  were  collectively  better  at  estimating  the  size  of  the  emplaced  ordnance 
than  they  were  at  the  class  of  target  detected.  For  demonstrators  providing  size  estimates,  about 

two-thirds  of  their  probabilities  of  classification  were  greater  than  0.5.  As  shown  in  Figure  6-8, 
demonstrators  were  not  as  successful  in  determining  class  as  size.  In  addition,  none  of  the  demonstrators 
were  able  to  correctly  classify  clusters,  even  though  many  of  them  could  detect  a  cluster. 
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62  ADDITIONAL  PERFORMANCE  STATISTICS 


One  goal  of  any  UXO  system  performance  assessment  is  to  determine  how  well  a  system  locates,  identifies, 
and  classifies  ordnance  for  remediation.  As  discussed  in  Section  4.1.4,  this  assessment  quantifies  a 
demonstrator’s  ability  to  provide  information  that  would  be  cost-effective  for  remediation.  Figure  6-9 
presents  a  scatter  plot  of  the  two  performance  statistics  for  each  demonstrator  probability  of  detection 
(ordnance)  and  false  alarm  ratio.  Because  of  the  relatively  poor  performance  in  discriminating  between 
ordnance  and  nonordnance,  all  demonstrator  declarations  were  treated  as  ordnance  detections  for  this  figure. 

Better  performance  is  found  in  the  upper  left  region  of  this  plot,  which  corresponds  to  the  highest  fraction  of 
ordnance  remediated  with  the  lowest  number  of  nonordnance  (or  empty)  holes  remediated.  As  with  the  PD 
versus  PFA  plot  (see  Figure  6-3),  the  three  demonstrators  with  the  highest  PD  values  (Parsons,  Geometries,  and 
Geophex)  surveyed  the  16A-hectare  area.  The  better  performing  demonstrators  from  the  16B-hectare  area 
generally  had  lower  fractions  of  detected  ordnance  remediated  and  much  higher  ratios  of  nonordnance  to 
ordnance  targets  remediated.  These  results  may  be  due  to  the  fact  that  the  16B -hectare  area  contained  more 
metallic  scrap  than  die  16A-hectare  area  (see  Appendix  D). 


As  described  in  Section  3.0,  a  number  of  plastic  mines  were  emplaced  in  the  16A-  and  the  16B-hectare  areas. 
Although  magnetometer  systems  were  not  expected  to  detect  these  mines,  some  demonstrators  claimed  that 
their  systems  had  this  detection  capability.  To  determine  whether  these  claims  were  valid,  demonstrators 
were  scored  separately  in  comparison  with  a  baseline  target  set  containing  these  mines.  Appendix  E  lists  the 
results  of  this  scoring.  All  of  the  measures  of  effectiveness  described  in  this  report,  with  the  exception  of 
those  in  Appendix  E,  were  computed  using  a  baseline  target  set  that  did  not  contain  plastic  mines.  None  of 
the  demonstrators,  except  Vallon,  had  PD  values  for  mines  that  were  statistically  different  from  the  PD  that 
would  be  obtained  from  a  random  report  of  targets. 

Vallon,  which  used  only  magnetometer  sensors,  detected  55  percent  (7  out  of  12)  of  the  mines.  As  discussed 
in  Appendix  E,  one  possible  reason  for  this  unexpectedly  high  detection  probability  is  that  Vallon’s  survey 
twn  observed  two  of  the  plastic  mines  that  migrated  to  the  surface.  This  situation  may  have  invited  closer 
scrutiny  of  that  area  during  the  survey  and  post-test  analysis. 
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63  OPERATIONAL  SUMMARY 

Systems  demonstrated  during  Phase  II  of  the  controlled  site  UXO-ATDs  had  various  implementation 

requirements.  The  demonstrators  also  had  different  support  needs,  both  on  site  and  locally.  Some  of  the 

systems  were  almost  entirely  self-sufficient,  while  others  depended  on  the  support  trailers  for  space  to 
perform  data  analysis.  All  of  the  systems  demonstrated  during  Phase  II  used  the  support  trailers  to  some 
degree,  although  if  the  trailers  had  not  been  available,  the  demonstrators  most  likely  would  have  provided 
their  own  support  equipment  In  any  case,  the  various  support  needs  must  be  considered  when  implementing 
these  systems  in  a  remote  location. 

In  addition  to  on-site  support,  the  demonstrators  also  used  local  supplies  and  services  to  varying  degrees. 
Some  demonstrators  shipped  all  of  their  equipment  to  JPG  or  transported  it  by  truck.  Several  of  the 
demonstrators  required  ATVs  that  were  either  rented  or  purchased  locally.  In  many  cases,  replacement  parts 
anH  tools  were  required  from  local  support  services  during  the  demonstrations.  Two  demonstrators 
experienced  equipment  breakdowns  that  required  machining  or  welding  for  repair.  In  one  instance,  a  tow 
truck  was  required  to  remove  a  survey  vehicle  that  became  stuck  in  the  mud  on  site.  The  local  support 
services  used  by  various  demonstrators  often  enabled  demonstration  activities  to  continue,  however,  in  a 
remote  location,  such  support  services  might  not  be  as  readily  available. 

The  capabilities  and  limitations  of  the  various  technologies  were  influenced  by  several  factors.  The  type  of 
system  demonstrated,  the  terrain  at  the  demonstration  areas,  and  the  weather  conditions  before  and  during  the 
demonstrations  all  affected  system  efficiency.  Man-portable  systems  were  the  most  successful  in  accessing 
all  areas  of  the  site,  because  these  systems  could  easily  maneuver  around  and  through  forested  areas. 
Vehicle-towed  systems  often  had  difficulty  driving  through  the  deeply  rutted  areas  of  the  demonstration  sites. 
Similarly,  densely  forested  areas  were  not  accessible  to  vehicle-towed  systems. 

The  airborne  system  demonstrated  by  Aerodat  was  affected  by  the  forested  terrain  in  that  the  tree  level 
prevented  Aerodat  from  flying  over  the  demonstration  area  at  its  optimal  elevation.  The  two  other  airborne 
demonstrators,  AES  and  SRI,  stated  that  their  GPR  systems  had  limited  capability  due  to  the  wet  ground 
conditions  during  the  demonstrations . 
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Several  man-portable  and  vehicle-towed  systems  experienced  difficulties  in  maintaining  the  GPS  satellite 
reception  during  the  demonstration  period,  primarily  in  areas  with  heavy  tree  cover. 


Both  remediation  systems  were  demonstrated  at  JPG  under  dry  conditions.  If  the  conditions  had  not  beat  dry, 
both  the  Wright  Laboratory  and  CEG  vehicles  could  have  become  immobilized  in  the  wet  heavy  clay  soils. 
Even  with  the  dry  ground  conditions,  CEG  had  to  avoid  deeply  rutted  or  low  areas  when  maneuvering  its  soft 
trencher,  which  is  top  heavy  and  unstable  when  traveling  on  uneven  terrain. 

In  general,  vehicle-towed  systems  were  more  prone  to  breakdowns  than  either  die  man-portable  or  the 
airborne  systems.  Demonstrators  that  brought  replacement  parts  were  better  prepared  for  field  operations 
than  those  that  had  to  have  equipment  or  parts  shipped  via  overnight  delivery.  The  two  remediation  systems 
experienced  numerous  equipment  breakdowns,  as  discussed  in  Section  5.0.  Depending  on  the  type  of  repairs 
required,  these  breakdowns  were  time-consuming,  and  resulted  in  lost  demonstration  time.  For  example,  the 
final  breakdown  for  CEG  ended  its  demonstration,  because  the  repair  could  not  be  made  in  the  field. 
Remediation  systems  experienced  difficulty  in  remote  operations. 

Areal  coverage  for  the  technologies  varied  widely.  Table  6-5  lists  the  coverage  (area  scored)  for  detection 
demonstrators.  Airborne  systems  provided  the  best  coverage,  as  might  be  expected,  but  their  coverage  rate 
was  negated  by  poor  detection  results.  The  combined  man-portable  and  vehicle-towed  systems  provided 
good  area  coverage,  with  three  out  of  the  four  demonstrators  covering  the  full  16  hectares  assigned.  Four  of 
the  six  man-portable  systems  covered  all  of  the  area.  Neither  of  the  vehicle-towed  systems  covered  the  entire 

area. 

The  ease  of  using  the  different  systems  varied  greatly.  For  the  most  part,  vehicle-towed,  airborne,  and 
remediation  systems  were  operated  by  experienced  personnel.  However,  both  Vallon  and  Scmtrex  used 
personnel  who  had  no  previous  experience  operating  the  equipment  Although  these  inexperienced  operators 
required  some  training,  they  were  able  to  work  independently  within  a  day  or  so.  The  quick  training  time  is 
significant  because  it  increases  the  potential  for  the  equipment’s  future  use  by  inexperienced  personnel. 
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TABLE  6-5 

SUMMARY  OF  DEMONSTRATOR  AREA  COVERAGE 


Demonstrator 

Coverage 

(percent) 

Demonstrator 

Coverage 

(percent) 

Demonstrator  • 

Coverage 

(percent) 

Aerodat 

100 

Geo-Centers 

100 

Parsons 

100 

AES 

100 

Geometries 

100 

Poles  tar 

93.8 

ADI 

100 

Geophex 

100 

SRI 

100 

Bristol 

83.9 

GeoPotendal 

84 

Scintrex 

36 

Coleman 

100 

Kaman 

52.6 

Vallon 

52.5  i 

Several  systems  included  real-time  data  analysis.  For  example.  Geometries  collected  additional  data  with  the 
GPR  system  in  areas  where  ordnance  was  detected  by  the  first  system  (although  data  collected  by  the  GPR 
system  was  not  included  for  data  analysis).  Such  real-time  data  analysis  also  permitted  the  demonstrators  to 
determine  whether  the  system  was  functioning  properly. 

As  expected,  the  various  man-portable,  vehicle-towed,  combined,  and  airborne  systems  each  had  specific 
capabilities  and  limitations.  While  the  man-portable  systems  proved  to  be  durable  and  were  able  to  access 
the  entire  site  successfully,  the  systems  were  limited  by  the  speed  and  stamina  of  the  operator.  The  vehicle- 
towed  systems  were  able  to  cover  the  site  quickly  but  were  often  subject  to  breakdowns  that  caused  time- 
consuming  delays. 

6.4  SUMMARY  AND  CONCLUSIONS 


Several  conclusions  can  be  drawn  from  die  performance  statistics.  In  general.  Phase  II  data  indicate  the 
following  trends  or  conclusions:. 


Magnetometer  and  EM  induction  sensors  outperformed  the  GPR  sensors. 

Air  demonstrators  continued  to  demonstrate  the  same  poor  performance  shown  by  Phase  I 
demonstrators. 

A  technology  demonstration  employing  several  different  sensor  types  can  potentially 
increase  performance  in  both  detection  and  classification  (see  Table  6-1). 

The  combined  sensor  systems  were  also  superior  to  either  system  alone  for  classifying 
ordnance  items  by  both  size  and  class. 
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A  wide  range  of  detection  performance  was  exhibited  by  the  Phase  II  demonstrators.  The  higher  PD  values  of 
certain  demonstrators  (compared  to  Phase  I PD  values)  may  be  due  to  several  factors,  including  (1)  the  use  of 
advanced  data  evaluation  techniques,  (2)  the  experience  level  of  the  demonstrators,  and  (3)  improved 
navigational  capability. 

With  respect  to  operational  analysis,  the  man-portable  systems  proved  to  be  durable  and  were  able  to  access 
most  areas;  however,  systems  were  limited  by  the  speed  and  stamina  of  the  operator,  the  climatic  conditions, 
and  the  terrain  Vehicle-towed  systems  covered  the  area  more  quickly  than  man-portable  systems,  but  they 
w ere  hindered  by  breakdowns  and  terrain.  The  combined  sensor  systems  took  advantage  of  the  strengths  of 
each  system  while  compensating  for  their  weaknesses.  Airborne  systems  had  the  best  coverage  and  speed  of 
all  the  systems,  but  they  lagged  far  behind  in  their  detection,  localization,  identification,  and  classification 
ability.  The  two  remediation  systems  sucessfully  demonstrated  their  ability  to  excavate  buried  ordnance, 
however,  both  systems  experienced  numerous  breakdowns,  operated  slowly,  and  had  difficulty  operating 
remotely. 


Several  conclusions  can  be  inferred  from  a  comparison  between  detection  abilities  of  the  various 
demonstrators  at  the  16A-  and  16B-hectare  areas.  PD  values  at  these  two  areas  are  generally  quite  similar, 
indicating  that  in  terms  of  detecting  ordnance,  demonstrators  at  both  areas  had  similar  performance 
capability.  However,  demonstrator  FAR  values  were  greater  at  the  16B-hectare  area,  most  likely  due  to  the 
larger  amount  of  nonordnance  clutter  present  at  the  site  (see  Appendix  D).  Demonstrators  at  the  16B-hectare 
area  identified  more  of  the  detected  ordnance  correctly  than  those  at  the  16A-hectare  area,  possibly  as  a  result 
of  the  previous  experience  demonstrators  gained  at  the  1 6  A-hectare  area  during  Phase  I. 


Because  the  specific  performances  of  systems  at  JPG  are  related  to  the  environmental  conditions  and  imposed 
operational  restrictions  at  the  site,  demonstration  results  at  JPG  may  not  be  directly  transferable  to  other  sites. 
However,  Government  and  private  industry  will  be  able  to  use  the  data  from  these  demonstrations  to  allocate 
their  resources  for  developing  more  effective  systems  for  defense-related  UXO  cleanup  efforts.  In  addition, 
environmental  restoration  managers  at  government  installations  will  have  an  independent  source  of 
information  that  can  help  them  identify  potentially  appropriate  and  cost-effective  technologies  given  the  site- 
specific  conditions  and  the  limitations  of  those  technologies. 
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Figure  6-1 

Individual  Demonstrator  Probability  of  Detection  (Ordnance) 


Demonstrator 


Figure  6-2 

Individual  Demonstrator  False  Alarm  Rate 


Demonstrator 


Figure  6-3 

Plot  of  Demonstrator  Probability  of  Detection  (Ordnance)  versus  Probability  of  False  Alarm 
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Probability  of  False  Alarm 


Figure  6-4 

Optimal  Performance  Statistics  by  Demonstrator 
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Parsons  Geophex  ADI  (MAG  Bristol  Valbn  GeoPotential  Aerodat  Kaman 

&  EM) 

Geometries  Geo-Centers  ADI  (MAG)  Scintrex  Coleman  AES  SRI 

Demonstrator 


Figure  6*5 

Individual  Demonstrator  Localization  Performance 
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□  Horizontal  Position  Error  (Radial)  ■  Absolute  Depth  Error 


Figure  6-6 

Individual  Demonstrator  Probability  of  Correct  Identification  (Classification  by  Type) 
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Notes:  1  Demonstrator  reported  all  target  declarations  as  “ordnance” 

2  Demonstrator  did  not  provide  type  information  for  declarations  or  listed  as  "unknown’ 


Figure  6-7 

Individual  Demonstrator  Probability  of  Correct  Classification  by  Size 
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Notes:  1  Demonstrator  did  not  provide  size  information  for  declarations  or  listed  as  “unknown' 


Figure  6-8 

Individual  Demonstrator  Probability  of  Correct  Classification  by  Class 
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□  Bomb  U  Projectile  ■  Mortar  E3  Cluster 


Figure  6-9 

Plot  of  Demonstrator  Probability  of  Detection  (Ordnance)  versus  False  Alarm  Ratio 
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False  Alarm  Ratio  (False  Alarms/Ordnance  Items  Detected) 
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APPENDIX  A 

PERFORMANCE  MEASUREMENT  ALGORITHM  DESCRIPTIONS 

Information  contained  in  this  appendix  was  compiled  from  documents  generated  by  the  Institute  for  Defense  Analysis  (IDA) 


This  appendix  describes  how  baseline  items  are  matched  with  demonstrator  declarations  to  determine  Pck»«t 
and  p  .  Demonstrator  declarations  at  locations  where  no  baseline  target  exists,  also  referred  to  as  false 
alarms,  are  computed  for  each  matching  method  The  following  terminology  is  used  when  describing 
performance  measurement  algorithms: 

•  Palling  item  —  An  in  rat  ordnance  or  nonordnance  object  emplaced  on  site  to  measure 
demonstrator  detection  capability 

•  Demonstrator  declaration  -  A  detection  reported  by  a  demonstrator 

•  -  Maximum  radius  of  the  circle  that  defines  a  group  of  baseline  items;  the  group 
radius  should  be  larger  than  the  expected  sensor  resolution. 

•  R^it  -  Maximum  horizontal  distance  allowed  between  a  baseline  item  and  a  demonstrator 

declaration  of  that  item  for  that  item  to  be  considered  detected;  is  intended  to  indicate 

the  demonstrator's  location  accuracy,  which  may  be  different  from  the  sensor  resolution,  R^t 
is  constrained  by  the  intended  size  of  excavation  during  remediation. 

COMPUTATION  OF 

The  P  i  ,i  algorithm  measures  demonstrator  performance  by  finding  one-to-one  matches  between  baseline 
itffmc  and  demonstrator  declarations.  As  the  name  implies,  this  algorithm  matches  each  baseline  item  with 
the  demonstrator  declaration  that  is  horizontally  closest  to  it.  Initially,  all  baseline  items  within  a  critical 
radius  of  a  demonstrator  declaration  are  identified.  R^*  is  used  to  define  a  circle  centered  on  each 

demonstrator  declaration.  A  tie  occurs  when  more  than  one  baseline  item  lies  within  die  circle.  These  ties  are 
broken  by  attempting  to  match  the  demonstrator  declaration  with  the  baseline  item  that  is  closest  to  it,  while 
simultaneously  maximizing  the  number  of  detected  baseline  items. 

To  start  the  matching  process,  all  demonstrator  declarations  are  reviewed  and  all  baseline  items  are 
considered  for  a  possible  match;  if  a  demonstrator  declaration  lies  within  R**  of  a  baseline  item,  that 
declaration  is  retained  as  a  potential  match.  All  potential  matches  are  retained  on  a  linked  list  associated  with 
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each  baseline  item.  An  algorithm  sorts  this  list  according  to  the  computed  distance  between  the  demonstrator 
declaration  and  the  baseline  item,  with  the  demonstrator  declaration  closest  to  the  baseline  item  listed  first, 
followed  by  the  second  closest  and  so  on.  After  this  first  iteration,  the  lists  arc  reviewed  to  make  sure  that 
different  baseline  items  are  not  matched  to  the  same  demonstrator  declaration.  If  the  same  demonstrator 
declaration  appears  as  the  first  entry  in  two  or  more  lists,  then  the  duplicate  demonstrator  declaration  is 
removed  from  all  but  one  of  the  lists. 

To  resolve  potential  duplicate  matches,  the  algorithm  compares  the  lists  of  each  baseline  item  to  the  lists  of 
all  die  other  baseline  items  in  a  series  of  two-by-two  comparisons.  If  two  baseline  items  have  the  same 
demonstrator  declaration  at  the  head  of  their  potential  matches  list,  three  situations  are  possible.  The 
situation  and  their  corresponding  solutions  are  given  below. 


Situation  1.  Both  baseline  items  have  potential  matches  with  more  than  one  demonstrator 
declaration  (that  is,  each  list  is  less  than  2). 


Solution:  Minimize  the  sum  of  the  distances  between  the  baseline  items  and  the  demonstrator 
declarations.  For  example,  in  the  following  situation  demonstrator  declaration  dl  could  be 
matched  to  either  baseline  item  A  or  B. 


•  Baseline  £.  Baseline  B 

dl  1.5 ft  dl  2.0ft 

d2  2.0ft  d3  3.0ft 

•  Option  1: 

Baseline  A — dl  distance  =  1.5ft 
Baseline  B—d3  distance  =  3.0ft 
sum  of  the  distances  =  4.5  ft 

•  Option  2: 

Baseline  A — d2  distance  =  2.0ft 
Baseline  B — dl  distance  =  2.0ft 
sum  of  the  distances  =  4.0  ft 

.  Option  2  produces  the  smaller  sum  of  the  distance.  Therefore,  dl  is  removed  from  the 

Baseline  A  list. 

Situation  2.  Only  one  baseline  item  has  more  than  one  potential  demonstrator  declaration  match. 

Solution:  The  baseline  item  that  has  only  one  potential  demonstrator  match  would  remain 
matched  to  its  single  demonstrator  declaration.  The  baseline  item  that  has  multiple  potential 
demonstrator  matches  would  remove  the  first  declaration  on  its  list. 
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Situation  3.  Neither  baseline  item  has  more  than  one  potential  match  to  a  demonstrator  declaration 
in  its  list  (that  is,  the  one  demonstrator  declaration  is  the  only  declaration  on  both  of  the  lists). 

Solution:  The  baseline  item  closest  to  the  demonstrator  declaration  is  matched  to  that 
declaration.  The  other  baseline  item  will  have  its  single  demonstrator  declaration  removed  from 
its  list  and  will  become  an  undetected  baseline  item. 


In  all  cases,  resolving  a  tie  removes  a  demonstrator  declaration  from  the  head  of  one  of  the  baseline  item 
potential  matches  list.  Notice  that  this  two-by-two  method  of  tie  breaking  does  not  globally  optimize  the 
distance  from  demonstrator  declaration  to  baseline  item  by,  for  example,  finding  all  baseline  items  with  the 
same  H^mnnctratnr  declaration  at  the  head  of  its  list  before  beginning  the  tie-breaking  process.  This  method 
favors  the  demonstrator  by  nearly  maximizing  the  number  of  matches. 

The  process  of  breaking  one  tie  may  create  another  tie.  Therefore,  the  method  of  searching  for  and  resolving 
ties  is  repeated  until  no  further  ties  are  found.  Eventually,  a  loop  through  all  baseline  items  will  be  free  of 
ties.  When  this  occurs,  the  process  is  complete,  and  the  first  declaration  on  each  baseline  item  list  is  the  final 
selected  match.  At  this  point,  Pd —  can  be  calculated  as  follows: 

_  Racftlinft  items  with  a  demonstrator  match 
dasat  Total  baseline  items  emplaced 

The  number  of  false  alarms  is  the  number  of  demonstrator  declarations  that  were  not  matched  to  a  baseline 
item  (IDA  1994). 

COMPUTATION  OF 

If  the  sensor  resolution  is  not  adequate  to  resolve  some  of  the  closer  spaced  baseline  items,  Pda*®  ma V  n°t 
accurately  describe  a  demonstrator’s  detection  capability.  A  more  accurate  measure  of  demonstrator 
performance  may  be  P,^. 

A  group  of  baseline  items  separated  by  a  distance  that  is  less  than  the  sensor  resolution  will  most  likely  be 
detected  as  only  one  object.  An  accurate  measure  of  detection  performance  should  take  into  account  the 
effect  of  sensor  resolution  if  the  test  involves  Ihe  detection  of  closely  spaced  objects.  The  P^  method  of 
computing  demonstrator  performance  counts  such  groups  as  only  one  "item"  when  computing  probabilities  of 
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detection.  It  credits  a  demonstrator  with  detecting  a  group  of  objects,  and  does  not  count  as  missed  the 
individual  baseline  items  within  a  group.  is  computed  by  dividing  the  number  of  groups  of  baseline 

ordnance  item*;  within  of  a  demonstrator  declaration  by  the  total  number  of  baseline  groups  emplaced  as 

follows: 

p  _  Number  of  groups  detected 
graap  Total  groups  emplaced 


In  Figure  A-l,  the  circle  represents  the  demonstrator  declaration  and  the  x's  represent  three  closely  spaced 
baseline  items.  In  this  case,  counts  one  detection  out  of  one  possibility  for  the  group. 

The  Ppoup  algorithm  consists  of  three  steps: 


Step  1.  Determine  demonstrator  matches  to  individual  baseline  items;  this  step  uses  the  P, 
algorithm  to  find  one-to-one  matches  with  individual  baseline  items. 


Step  2.  Group  the  baseline  items  according  to  a  grouping  radius.  This  step  assembles  groups  of 
hackling  items  according  to  the  grouping  radius  (R^wp)-  A  group  can  consist  of  one  or  more  baseline 
items.  The  grouping  algorithm  initially  places  each  baseline  item  into  a  group  of  its  own— that  is 
each  baseline  item  is  considered  a  group.  Each  baseline  item  is  then  visited  in  turn,  and  all  other 
baseline  items  are  considered  for  inclusion  in  the  initial  baseline  item's  group.  The  first  baseline  item 
is  denoted  as  the  offering  target  and  the  second  baseline  item  is  the  considered  target.  If  the 
considered  target  falls  within  of  the  offering  target,  the  considered  target  is  added  to  the 
offering  target's  list  of  potential  group  members.  The  offering  target  considers  all  other  baseline 
items  for  inclusion  in  its  group,  creating  a  list  of  all  potential  group  candidates.  The  algorithm  then 
checks  the  list  for  candidates  that  have  previously  been  included  in  another  group  (the  single  item 
group  itself).  Candidate*:  included  in  another  group  are  removed  from  the  list;  those  that  are  not  are 
permanently  added  to  the  offering  target's  group. 


The  algorithm  then  rechecks  all  the  baseline  items  to  ensure  that  any  baseline  items  in  a  group  by 
themselves  are  not  within  R^  of  another  baseline  item.  If  a  lone  baseline  item  is  found  within  R^ 
of  another  baseline  item,  it  is  added  to  the  other  baseline  item's  group  unless  the  addition  increases 
beyond  the  radius  of  the  circle  that  encompasses  all  of  the  baseline  items  belonging  to  that 

group. 

Situations  may  occur  in  which  delineations  between  baseline  item  groups  are  not  clear.  For  example, 
two  groupings  for  the  same  baseline  item  set  are  shown  in  Figures  A-2  and  A-3.  All  four  baseline 
itemc  in  the  highlighted  area  cannot  be  included  in  the  same  group,  because  the  radius  of  the  circle 
encompassing  such  a  group  exceeds  the  The  figures  show  two  possible  ways  to  break  these 
baseline  items  into  two  groups.  Unfortunately,  the  single  baseline  item  difference  in  the  grouping 
will  change  the  measure  of  demonstrator  performance.  In  the  example  shown  in  Figure  A-2,  the 
demonstrator  is  credited  with  one  detection  and  one  false  alarm  because  both  detections  are  applied 
to  the  same  group. 
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Step  3.  Translate  individual  matches  found  in  the  first  step  to  group  matches  based  on  the  grouping 
produced  in  Step  2.  This  step  in  computing  Pg^  involves  counting  the  number  of  groups  detected 
using  the  results  of  the  first  two  steps.  If  a  demonstrator  has  declared  an  ordnance  item  with  Ra*  of  a 
baseline  item,  then  die  demonstrator  is  credited  with  finding  the  group  that  contains  that  baseline 
item,  flnH  group  is  marked  as  found.  If  the  group  has  already  been  marked  as  found,  any  further 
of  that  group  are  not  counted.  Because  of  die  way  the  algorithm  is  implemented,  the 
demonstrator  declaration  may  not  be  within  R^  of  the  average  (x,  y)  location  of  the  baseline  items  in 
the  group.  The  number  of  false  alarms  is  the  number  of  demonstrator  declarations  not  within  Ra*  of 
a  baseline  item  group. 
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Figure  A-l .  Two  methods  of  scoring  three  baseline  targets  within  a  2  meter  critical  radius  or  one  demonstrator  declaration. 


Figure  A-2.  An  example  of  an  ambiguous  grouping  that  results  in  a  decrease  in  demonstrator  probability  of  detection. 
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Figure  A-3.  An  example  of  a  second  grouping  of  the  same  baseline  shown  in  Figure  A-2. 


APPENDIX  B 

FENCE  LINE  AREA  EXTRACTIONS 
16A-  AND  16B-HECTARE  AREAS 

Plotted  target  reports  obtained  from  some  demonstrators  (including  ADI,  Geo- Centers,  and  Coleman)  showed 
fairly  straight  lines  typical  of  buried  pipelines  or  fences.  Subsequent  excavations  revealed  that  these  lines 
correspond  to  buried  or  partially  buried  fence  wire.  The  presence  of  the  fence  wire  made  it  necessayr  to 
exclude  from  scoring  all  demonstrator  data,  as  well  as  baseline  targets,  within  a  specified  distance  of  the 
fence  «wgm<»ntg  This  approach  ensured  that  demonstrators  were  not  penalized  when  they  detected  the  fence. 

To  exclude  the  fence  line  areas,  fence  segments  in  the  16A-  and  16B-  hectare  areas  were  first  identified  using 
ADFs  data  The  size  of  the  area  around  each  fence  segment  was  then  determined  to  exclude  fence  line  data 
from  the  scoring  process.  For  this  step,  the  data  were  examined,  and  rectangles  were  centered  about  each  of 
die  fence  segments.  Since  some  scatter  was  observed  in  the  points  describing  the  fence  segments,  the  widths 
of  these  rectangles  were  adjusted  until  all  of  the  data  points  appeared  to  be  within  the  rectangles.  This 
process  resulted  in  a  4-meter-wide  rectangular  area  of  exclusion. 

After  all  demonstrator  reports  outside  the  rectangles  were  excluded  from  the  data  set,  the  residual  variance  of 
the  points  about  the  fitted  lines  was  computed  for  all  fence  segments.  The  standard  deviations  ranged  from 
0.7  to  1.0  meter.  About  2  standard  deviations  (2  meters)  on  either  side,  or  a  4-meter-wide  rectangle,  will 
include  95  percent  of  the  points. 

The  above  analysis  acc^mfe  for  only  the  fence  line  segments  and  any  deviations  from  the  theoretical  straight  line 
drawn  to  subdivide  the  rectangles.  However,  when  scoring  each  of  the  demonstrators,  a  critical  radius  of  2  meters 
was  centered  about  each  of  the  baseline  targets  to  account  for  factors  such  as  errors  in  navigation  or  sensor 
response.  For  example.  Parsons  stated  that  the  output  of  its  sensor  is  influenced  by  surface  objects,  such  as  wire 
fences,  located  within  a  3-  to  4-meter  radius  of  the  sensor.  As  a  result,  demonstrator  reports  that  identify  target 
locations  near  the  sides  of  these  rectangles  may  result  from  portions  of  the  fence  that  are  within  this  critical 
radius.  To  ensure  that  these  false  reports  are  not  sccxd,  the  sides  of  the  rectangles  were  increased  by  an  additional 
2  meters  (the  critical  radius)  on  either  side  of  the  fence  for  a  total  rectangle  width  of  8  meters. 
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When  the  wiring  algorithm  was  applied  to  the  demonstrator  data,  any  baseline  targets  or  demonstrator  data 
falling  within  the  rectangles  were  not  considered  in  the  performance  assessment  To  exclude  these  targets  and 
data,  the  location  (x^y*)  of  each  report  and  baseline  item  was  filtered  to  see  if  it  met  the  following  criteria. 


min  (Xv  X>)  -  4.0  max  (Xv  X2)  +  4.0 

min  (Tj,  YJ  "  4  0  *  Yt  5  max  (Tj,  YJ  +  4.0 


Points  meeting  these  criteria  were  then  tested  to  determine  if  they  were  horizontally  within  4  meters  of  the 
straight  line  fence  equation  (y  =  mx  +  b)  described  above.  The  perpendicular  distance  (d  j.)from  any  point 
(xtjy,)  to  a  straight  can  then  be  determined  if  the  line  fits  of  the  following  form: 

Ax+By  +  C  =  0 


The  perpendicular  distance  was  calculated  using  the  following  equation: 

d±  =  Ax'+9y>  +  C 

2+B2 


where 

A  =  m 
B  =  -1 
C  =  b 


If  dx  was  less  than  or  equal  to  4.0,  the  point  was  excluded  from  consideration  in  the  performance  assessment 
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APPENDIX  C 
DERIVATION  OF 

Assessing  the  detection  performance  of  any  system  requires  considering  not  only  how  well  it  detects  the 
desired  UXO  targets,  but  also  how  many  false  alarms  occur  in  the  process.  A  system  with  a  high  probability 
of  detection  and  a  high  rate  of  false  alarms  would  probably  not  be  feasible  for  use  in  the  field  because  for 
every  true  target  found  and  remediated,  excessive  resources  (such  as  time,  money,  and  personnel)  would  be 
expended  to  remediate  the  false  detections.  The  most  desirable  systems  have  both  high  probabilities  of 
detection  and  low  false  alarm  rates. 


The  detection  probabilities  of  the  various  demonstrator  systems  can  be  quantified  accurately  based  cm  the 
large  number  of  emplaced  targets.  However,  quantification  of  the  false  alarm  rate  for  these  systems  presents 
problems.  Typically,  in  a  controlled  test,  any  sensor  responses  not  associated  with  die  known  baseline  target 
set  are  declared  as  false  alarms.  This  approach  assumes  that  the  rest  of  the  field  is  free  from  UXO  objects.  A 
large  percentage  of  demonstrator  reports  not  associated  with  baseline  targets  on  the  16B -hectare  (40B-acre) 
area  corresponded  to  various  man-made  items  (such  as  plow  points  or  wire)  and  UXO  items  (some  live)  that 
were  not  emplaced  for  this  test  Because  of  these  extraneous  items,  false  alarm  rates  cannot  be  accurately 
quantified  without  remediating  a  large  number  of  each  demonstrator's  reports. 

However,  a  comparison  of  the  number  of  reported  targets  from  the  various  demonstrators  shows  a  wide  range 
in  the  number  of  potential  targets.  Figures  C-l,  C-2,  and  C-3  present  plots  of  the  number  of  target  reports  by 
demonstrator  for  the  16A-hectare  (40A-acre),  16B-hectare  (40B-acre),  and  32-hectare  (80-acre)  areas, 
respectively.  In  these  figures,  the  numbers  differ  by  an  order  of  magnitude  within  the  same  site  and,  in  many 
inctaprpg  with  the  same  type  of  sensor  (for  example,  magnetometer).  Assuming  similar  test  conditions,  the 
question  remains,  "Why  are  the  number  of  reports  so  different? 

For  demonstrators  with  large  numbers  of  reports,  possible  causes  must  be  examined.  One  possible  cause  for 
the  reports  may  be  that  lower  noise  levels  allow  more  of  the  smaller  or  deeper  targets  to  be  detected.  The 
lower  noise  levels  could  be  due  to  lowered  sensor  levels,  better  signal  processing,  or  more  advanced  post¬ 
processing  of  the  data.  Higher  noise  levels  (due  to  sensor  noise,  electromagnetic  interference,  or  motion 
contamination)  may  also  result  in  more  false  reports  due  to  spurious  noise  spikes,  particularly  if  the  noise 
were  not  Gaussian.  This  latter  case  is  addressed  in  the  following  derivation. 
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If  the  number  of  demonstrator  reports  is  assumed  to  be  due  to  random  noise,  the  number  of  targets  that  would 

be  detected  randomly  must  be  determined.  To  quantify  this  hypothesis,  a  measure  of  effectiveness, 

was  developed.  represents  the  probability  of  detection  due  to  the  random  distribution  of  demonstrator 

reports. 

To  develop  this  measure,  the  site  area  ,4  is  divided  into  individual  cells  with  an  area  equal  to  JtR2c,  as  defined 
by  the  critical  radius  R^  For  a  single  random  declaration,  the  probability  of  hitting  a  specific  cell  is  equal  to 
jdPjA .  This  process  is  repeated  for  die  number  of  trials  («),  which  corresponds  to  the  number  of 
demonstrator  reports.  Experiments  of  this  type  have  a  wide  range  of  application  and  are  known  in  probability 
theory  as  Bernoulli  trials.  Bernoulli  trials  are  defined  as  repeated  independent  trials  with  only  two  outcomes 
(for  example,  hit  or  miss)  for  each  trial,  and  their  respective  probabilities  remain  constant  throughout  the 
trials  (Feller  1968).  After  n  trials,  a  particular  cell  may  contain  from  Oton  reports.  According  to  Fella-,  the 
probability  of  a  cell  containing  exactly  £  reports  after  n  Bernoulli  trials  is  denoted  by  b(k:n,p)  and  is  defined 
by  die  following  equation: 


b(t.nj>)  - 


where 


n\ 

(n-k)\k\ 


nRc 

p  =  Probability  of  hit  =  — — 
q  -  Probability  of  miss  =  1  -p 

Using  die  above  equation,  the  probability  of  having  no  reports  in  a  particular  cell  (k  -  0)  is  <f,  and  the 
probability  of  having  at  least  one  report  within  a  cell  is  1-  <f.  This  equation  is  referred  to  as  the  binomial 

distribution. 
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For  most  of  the  cases  considered  in  this  report,  n  is  very  large  (typically  100  to  2,000),  and/?  is  very  small 
(0.00008).  In  these  situations,  it  is  convenient  to  use  the  Poisson  approximation  of  the  binomial  distribution. 
The  Poisson  approximation  for  the  probability  of  a  cell  containing  exactly  k  reports  after  n  Bernoulli  trials  is 
calculated  using  the  following  equation: 


/KM-)  *  b(fcnj>) 


The  probability  of  having  no  reports  in  a  particular  cell  (k  =  0)  is  e\  and  the  probability  of  having  at  least 
one  report  within  a  cell  is  which  is  equal  to  P,****,- 
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Number  of  Reports 


FIGURE  C-l 

16A-HECTARE  (40A-ACRE)  AREA 


Bristol  Geometries  Geophex  GeoPotential  Kaman  Parsons 


Demonstrator 
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FIGURE  C-3 

32-HECTARE  (80-ACRE)  AREA 
TARGET  REPORTS  BY  DEMONSTRATOR 


Aerodat 


AES 

Demonstrator 
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APPENDIX  D 

JPG  CONTROLLED  SITE  16B-HECTARE  DEMONSTRATION  AREA 
TARGET  ANOMALY  INVESTIGATION 

During  Phase  I  of  the  UXO  ATD  controlled  site  test,  the  16A-hectare  area  was  surveyed  by  ground-based 
systems,  and  the  32-hectare  area  was  surveyed  by  airborne  systems.  Locations  where  several  of  the 
demonstrators  had  coincidental  reports  were  remediated  to  determine  whether  the  repents  were  accurate.  The 
32-hectare  area  was  not  remediated  due  to  the  sparse  number  of  reports  from  the  airborne  systems. 

During  Phase  H,  a  portion  of  the  32-hectare  area  (the  16B-hectare  area)  was  set  aside  for  ground-based 
surveys.  On  the  16B-hectare  area,  a  large  number  of  demonstrator  reports  were  not  aligned  with  known 
baseline  targets.  To  investigate  whether  these  reports  corresponded  to  real  objects,  a  limited  number  of  report 
locations  were  remediated 


For  this  investigation,  the  four  ground  demonstrators  from  the  16B -hectare  area  were  selected.  Next, 
locations  were  selected  if  at  least  three  of  the  four  ground  demonstrators  reported  targets  within  a  2-meter 
radius  of  each  other  but  not  within  a  2-meter  radius  of  any  known  baseline  object  This  analysis  yielded 
42  holes  that  were  subsequently  remediated  Of  these  42  holes,  38  contained  man-made  debris,  and  two 
holes  contained  live  UXO  items;  in  all,  40  holes  (or  95.2  percent)  contained  metallic  items.  The  results  for 
each  remediation  are  listed  in  Table  D-l. 

Because  of  the  high  yield  from  this  remediation,  16  additional  reports  were  selected  using  the  following 
criteria:  the  individual  demonstrator  reports  were  (1)  isolated  (that  is,  not  within  a  2-meter  radius  of  the  other 
demonstrators)  and  (2)  not  within  a  2-meter  radius  of  a  known  baseline  object.  Of  these  16  single  reports, 
nine  holes  yielded  man-made  objects,  and  one  hole  yielded  an  ordnance  item;  in  all,  1 0  holes  (or  62.5  percent) 
contained  metallic  items. 

Based  on  these  results,  the  16B-hectare  area  appears  to  be  cluttered  with  man-made  debris.  As  a  result,  the 
definition  of  false  alarm  was  modified  to  include  only  those  reports  not  associated  with  ordnance  items.  The 
ordnance  items  found  were  located  in  the  western  portion  of  the  area  near  the  edge  of  the  firing  range.  Due  to 
the  location  and  small  percentage  of  UXO  items  found  during  this  remediation,  the  number  of  nonbaseline 
UXO  items  still  within  this  area  is  believed  to  be  negligible. 
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TABLE  D-l 

16B-HECTARE  DEMONSTRATION  AREA 
TARGET  ANOMALY  INVESTIGATION 
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099  At  least  three  640710.340  4304504.550  11/14/95  0.30  One  metal  bracket 

demonstrators  detected 
the  target. 


TABLE  D-l 

16B-HECTARE  DEMONSTRATION  AREA 
TARGET  ANOMALY  INVESTIGATION 
(CONTINUED) 
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TABLED-! 

16B-HECTARE  DEMONSTRATION  AREA 
TARGET  ANOMALY  INVESTIGATION 
(CONTINUED) 
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TABLE  D-l 

16B-HECTARE  DEMONSTRATION  AREA 
TARGET  ANOMALY  INVESTIGATION 
(CONTINUED) 
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Only  Geo-Centers  640780.300  4303938.300  1 1/17/95  NA  No  target  anomalies  detected  Within  a  1 .22-meter  radius,  no 

detected  the  target.  metallic  objects  were  detected. 


TABLE  D-l 

16B-HECTARE  DEMONSTRATION  AREA 
TARGET  ANOMALY  INVESTIGATION 
(CONTINUED) 


D-9 


APPENDIX  E 

MINE  DETECTION  STATISTICS 


Mine  detection  capabilities  were  not  the  focus  of  the  Phase  II UXO  ATDs;  however,  both  the  16A-  and 
]  6B-hectare  areas  contained  small  mine  fields.  The  1 6 A -hectare  area  contained  15  emplaced  mines,  and  the 
16B-bectare  area  contained  12  mines. 


Demonstrator  mine  detection  capabilities  are  shown  in  Table  E-l;  only  three  demonstrators  detected  mines. 
Parsons  Engineering  Science,  Inc.;  Bristol  Aerospace  Ltd.;  and  Vallon  GmbH  (Vallon).  Vallon  had  the 
higher  probability  of  detection,  most  likely  because  two  mines  were  visually  located  on  the  surface  of  the 
demonstration  area  during  the  course  of  the  survey. 
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TABLE  E-l 

MINE  DETECTION  STATISTICS 


Demonstrator 

Area  Surveyed 
(Hectares) 

No.  of 
Mines 

No.  of  Mines 
Detected 

Probability  of 
Detection  (PD) 

Aerodat  Inc. 

32 

12 

0 

0.00 

Airborne  Environmental 
Surveys 

32 

12 

0 

0.00 

Australian  Defense 
Industries,  Pty.  Inc. 

16B 

12 

0* 

0b 

0.00 

0.00 

- - - 

Bristol  Aerospace  Ltd. 

16A 

15 

3 

0.20 

Coleman  Research 
Corporation 

16B 

12 

0 

0.00 

Geo-Centers,  Inc. 

16B 

12 

0 

0.00 

Geometries,  Inc. 

16A 

15 

0 

0.00 

Geophex  Ltd. 

16A 

15 

0 

0.00 

Geopotential 

16A 

15 

0 

0.00 

Kaman  Sciences 
Corporation 

16A  | 

15 

0 

0.00 

Parsons  Engineering 
Science,  Inc. 

16A 

15 

1 

0.07 

Polestar  Technologies, 

Inc. 

16A 

15  ! 

NR 

NR 

SRI  International 

32 

12 

0 

0.00 

Scintrex,  Inc. 

16A 

12 

0 

0.00 

Vallon  GmbH 

16B 

12 

T 

0.58 

Notes: 


a  Electromagnetic  survey  results 

b  Magnetometer  survey  results 

c  Two  mint*  were  visually  located  by  Vallon  during  the  demonstration. 
NR  No  data  reported 
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APPENDIX  F 
DEMONSTRATOR  COST 

Table  F-l  presents  the  costs  of  surveying  a  hectare  of  land  at  oneofthe  three  JPG  control  sites.  Cost  data 
were  determined  by  dividing  the  individual  demonstrator  cost  by  the  number  of  hectares  die  demonstrator 
surveyed.  Remediation  demonstrator  costs  per  hectare  were  not  calculated  because  those  demonstrators 
remediated  selected  targets,  rather  than  a  specific  area.  In  addition,  because  the  Wright  Laboratory  REVS 
demonstration  was  performed  by  a  government  agency,  no  cost  information  was  provided. 


F-l 


APPENDIX  F 
DEMONSTRATOR  COST 


Demonstration 

Area  Scored 

Cost 

per 

Demonstrator 

Area 

(Hectares) 

Hectares 

Acres 

Cost 

(S) 

Hectare 

($) 

Airborne  Systems: 

Airborne  Environmental  Surveys 

32 

32 

80 

352,935 

11,029 

Aerodatlnc. 

32 

32 

80 

39,894 

1,250 

SRI  International 

32 

32 

80 

137,240 

4,289 

Ground  Systems: 

Australian  Defence  Industries,  Pty.  Inc. 

16B 

16 

40 

90,002 

5,625 

Geophex  Ltd. 

16A 

16 

40 

58,457 

3,529 

GeoPotential 

16A 

13.6 

33.6 

25,990 

1,911 

Parsons  Engineering  Science,  Inc. 

16A 

16 

40 

54,065 

3,379 

Polestar  Technologies,  Inc. 

16A 

16 

40 

166,293 

10,393 

Sdntrex,  Inc. 

16A 

5.8 

14.2 

41,643 

7,180 

Vehicle  Systems: 

Bristol  Aerospace  Ltd. 

16A 

13.6 

33.6 

99,806 

7,339 

Kaman  Sciences  Corporation 

16A 

8.5 

21 

245,285 

28,587 

Combined  Systems: 

Coleman  Research  Corporation 

16B 

16 

40 

78,862 

4,929 

Geo-Centers,  Inc. 

16B 

16 

40 

151,819 

9,489 

Geometries,  Inc. 

16A 

16 

40 

296,997 

18,562 

Vallon  GmbH  ( in  cooperation  with 

Security  Search  Products) 

16B 

16 

40 

66,308 

4,144 

Remediation: 

Concept  Engineering  Group 

16A 

1 

1 

70,459 

NA 

Wrigfat  Laboratory  REVS 

16A 

1 

1 

2 

NA 

Notes: 

1  Remdiation  demonstrations  not  scored. 

2  Government  demonstration  cost  not  provided. 


NA  not  available 
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GLOSSARY 


Abbreviations  and  Acronyms 


ADI 

Aerodat 

AES 

AETC 

AGPR 

ATD 

ATV 

ANSI 

Bristol 

Coleman 

CEG 

DAIS 

DRA 

DWP 

EM 

FAR 

Geo-Centers 

GER 


Geometries 

Geophex 

GDE 

GIS 

GPR 

GPS 

HEAT 

IDNR 

IDA 

JPG 

Kaman 

ManPODS 

MAPS 

MCS 

MOE 

NAVEODTECHDIY 

NEPA 

OPTEVFOR 

PBS 


Australian  Defence  Industries  Pty  Ltd. 
Aerodat,  Inc. 

Airborne  Environmental  Surveys 
Arete  Engineering  Technologies  Corporation 
Airborne  ground  penetrating  radar 
Advanced  Technology  Demonstration 
All-terrain  vehicle 

Amercian  National  Standards  Institute 
Bristol  Aerospace  Limited 
Coleman  Research  Corporation 
Concept  Engineering  Group,  Inc. 

Digital  airborne  imaging  spectrometer 
Demonstrator  reference  area 
Demonstration  work  plan 
Electromagnetic 
False  alarm  rate 
Geo-Centers,  Inc. 

Geophysical  and  Environmental  Research 

Corporation 

Geometries,  Inc. 

Geophex,  Ltd. 

GDE  Systems,  Inc. 

Geographic  information  system 

Ground-penetrating  radar 

Global  positioning  system 

High  Explosive  Anti-Tank 

Indiana  Department  of  Natural  Resources 

Institute  for  Defense  Analysis 

Jefferson  Proving  Ground 

Kaman  Sciences  Corporation 

Man-portable  detection  system 

Modular  azimuth  positioning  system 

Mobile  command  station 

Measures  of  effectiveness 

Naval  Explosive  Ordnance  Disposal 

Technology  Division 

National  Environmental  Policy  Act 

Operational  Test  and  Evaluation  Forces 

Precision  beacon  system 
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PC 

Parsons 

Polestar 

PRC 

projo 

PVC 

QA 

QAPP 

QC 

REVS 

SAR 

SEPOS 

SRI 

SHERP 

STOLS 

TAMIR 

TDEM 

TMA 

ToMAS 

USAEC 

USDA 

USGS 

UTM 

UXO 

Vallon 

Units  and  Measures 

°C 

°F 

ft 

hr. 

Hz 

in. 

kg 

kHz 

km 

lb. 

m 

MHz 

mm 


Personal  computer 

Parsons  Engineering  Science,  Inc. 

Polestar  Technologies,  Inc. 

PRC  Environmental  Management,  Inc. 

Projectile 

Polyvinyl  chloride 

Quality  assurance 

Quality  assurance  program  plan 

Quality  control 

Remote  excavation  vehicle  system 
Synthetic  aperture  radar 
Sensor  positioning  system 
SRI  International 

Safety,  Health,  and  Emergency  Response 
Plan 

Surface-towed  ordnance  locator  system 

Target  adaptive  matched  illumination  radar 

Time  domain  electromagnetic 

Target  matching  algorithm 

Towed  multi-sensor  array  system 

U.S.  Army  Environmental  Center 

U.S.  Department  of  Agriculture 

U.S.  Geological  Survey 

Universal  transverse  mercator 

Unexploded  ordnance 

Vallon  GmbH 


degrees  Centigrade 

degrees  Fahrenheit 

feet 

hour 

hertz 

inch(es) 

kilogram 

kilohertz 

kilometer 

pound 

meter 

megahertz 

millimeter 
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